Simple Network of Workstations

Simple Network of Workstations [R logo]

Help pages for package ‘snow’ version 0.3-3

clusterApply | Cluster-Level SNOW Functions
clusterApplyLB | Cluster-Level SNOW Functions
clusterCall | Cluster-Level SNOW Functions
clusterEvalQ | Cluster-Level SNOW Functions
clusterExport | Cluster-Level SNOW Functions
clusterMap | Cluster-Level SNOW Functions
clusterSetupRNG | Uniform Random Number Generation in SNOW Clusters
clusterSetupRNGstream | Uniform Random Number Generation in SNOW Clusters
clusterSetupSPRNG | Uniform Random Number Generation in SNOW Clusters
clusterSplit | Cluster-Level SNOW Functions
getMPIcluster | Starting and Stopping SNOW Clusters
makeCluster | Starting and Stopping SNOW Clusters
makeMPIcluster | Starting and Stopping SNOW Clusters
makeNWScluster | Starting and Stopping SNOW Clusters
makePVMcluster | Starting and Stopping SNOW Clusters
makeSOCKcluster | Starting and Stopping SNOW Clusters
parApply | Higher Level SNOW Functions
parCapply | Higher Level SNOW Functions
parLapply | Higher Level SNOW Functions
parMM | Higher Level SNOW Functions
parRapply | Higher Level SNOW Functions
parSapply | Higher Level SNOW Functions
setDefaultClusterOptions | Starting and Stopping SNOW Clusters
stopCluster | Starting and Stopping SNOW Clusters


Cluster-Level SNOW Functions

Description

Functions for computing on a SNOW cluster.

Usage

clusterSplit(cl, seq)
clusterCall(cl, fun, ...)
clusterApply(cl, x, fun, ...)
clusterApplyLB(cl, x, fun, ...)
clusterEvalQ(cl, expr)
clusterExport(cl, list)
clusterMap(cl, fun, ..., MoreArgs = NULL, RECYCLE = TRUE)

Arguments

cl | cluster object
fun | function or character string naming a function
expr | expression to evaluate
seq | vector to split
list | character vector of variables to export
x | array
... | additional arguments to pass to standard function
MoreArgs | additional argument for fun
RECYCLE | logical; if true shorter arguments are recycled

Details

These are the basic functions for computing on a cluster. All evaluations on the slave nodes are done using tryCatch. Currently an error is signaled on the master if any one of the nodes produces an error. More sophisticated approaches will be considered in the future.

clusterCall calls a function fun with identical arguments ... on each node in the cluster cl and returns a list of the results.

clusterEvalQ evaluates a literal expression on each cluster node. It a cluster version of evalq, and is a convenience function defined in terms of clusterCall.

clusterApply calls fun on the first cluster node with arguments seq[[1]] and ..., on the second node with seq[[2]] and ..., and so on. If the length of seq is greater than the number of nodes in the cluster then cluster nodes are reqycled. A list of the results is returned; the length of the result list will equal the length of seq.

clusterApplyLB is a load balancing version of clusterApply. if the length p of seq is greater than the number of cluster nodes n, then the first n jobs are placed in order on the n nodes. When the first job completes, the next job is placed on the available node; this continues until all jobs are complete. Using clusterApplyLB can result in better cluster utilization than using clusterApply. However, increased communication can reduce performance. Furthermore, the node that executes a particular job is nondeterministic, which can complicate ensuring reproducibility in simulations.

clusterMap is a multi-argument version of clusterApply, analogous to mapply. If RECYCLE is true shorter arguments are recycled; otherwise, the result length is the length of the shortest argument. Cluster nodes are recycled if the length of the result is greater than the number of nodes.

clusterExport assigns the global values on the master of the variables named in list to variables of the same names in the global environments of each node.

clusterSplit splits seq into one consecutive piece for each cluster and returns the result as a list with length equal to the number of cluster nodes. Currently the pieces are chosen to be close to equal in length. Future releases may attempt to use relative performance information about nodes to choose split proportional to performance.

For more details see http://www.stat.uiowa.edu/~luke/R/cluster/cluster.html.

Examples

  ## Not run:
cl <- makeSOCKcluster(c("localhost","localhost"))

clusterApply(cl, 1:2, get("+"), 3)

clusterEvalQ(cl, library(boot))

x<-1
clusterExport(cl, "x")
clusterCall(cl, function(y) x + y, 2)

  ## End(Not run)
[Package snow version 0.3-3 Index]


Higher Level SNOW Functions

Description

Parallel versions of apply and related functions.

Usage

parLapply(cl, x, fun, ...)
parSapply(cl, X, FUN, ..., simplify = TRUE, USE.NAMES = TRUE)
parApply(cl, X, MARGIN, FUN, ...)
parRapply(cl, x, fun, ...)
parCapply(cl, x, fun, ...)
parMM(cl, A, B)

Arguments

cl | cluster object
fun | function or character string naming a function
X | array to be used
x | matrix to be used
FUN | function or character string naming a function
MARGIN | vector specifying the dimensions to use.
simplify | logical; see sapply
USE.NAMES | logical; see sapply
... | additional arguments to pass to standard function
A | matrix
B | matrix

Details

parLapply, parSapply, and parApply are parallel versions of lapply, sapply, and apply.

parRapply and parCapply are parallel row and column apply functions for a matrix x; they may be slightly more efficient than parApply.

parMM is a very simple(minded) parallel matrix multiply; it is intended as an illustration.

For more details see http://www.stat.uiowa.edu/~luke/R/cluster/cluster.html.

Examples

  ## Not run:
cl <- makeSOCKcluster(c("localhost","localhost"))
parSapply(cl, 1:20, get("+"), 3)

## End(Not run)
[Package snow version 0.3-3 Index]


Uniform Random Number Generation in SNOW Clusters

Description

Initialize independent uniform random number streams to be used in a SNOW cluster. It uses either the L'Ecuyer's random number generator (package rlecuyer required) or the SPRNG generator (package rsprng required).

Usage

clusterSetupRNG (cl, type = "RNGstream", ...)

clusterSetupRNGstream (cl, seed=rep(12345,6), ...)
clusterSetupSPRNG (cl, seed = round(2^32 * runif(1)),
                   prngkind = "default", para = 0, ...)

Arguments

cl | Cluster object.
type | type="RNGstream" (default) initializes the L'Ecuyer's RNG. type="SPRNG" initializes the SPRNG generator.
... | Arguments passed to the underlying function (see details bellow).
seed | Integer value (SPRNG) or a vector of six integer values (RNGstream) used as seed for the RNG.
prngkind | Character string naming generator type used with SPRNG.
para | Additional parameters for the generator.

Details

clusterSetupRNG calls (subject to its argument values) one of the other functions, passing arguments (cl, ...). If the "SPRNG" type is used, then the function clusterSetupSPRNG is called. If the "RNGstream" type is used, then the function clusterSetupRNGstream is called.

clusterSetupSPRNG loads the rsprng package and initializes separate streams on each node. For further details see the documentation of init.sprng. The generator on the master is not affected.

clusterSetupRNGstream loads the rlecuyer package, creates one stream per node and distributes the stream states to the nodes.

For more details see http://www.stat.uiowa.edu/~luke/R/cluster/cluster.html.

Examples

  ## Not run:
clusterSetupSPRNG(cl)
clusterSetupSPRNG(cl, seed=1234)
clusterSetupRNG(cl, seed=rep(1,6))

## End(Not run)
[Package snow version 0.3-3 Index]


Starting and Stopping SNOW Clusters

Description

Functions to start and stop a SNOW cluster and to set default cluster options.

Usage

makeCluster(spec, type = getClusterOption("type"), ...)
stopCluster(cl)

setDefaultClusterOptions(...)

makeSOCKcluster(names, ..., options = defaultClusterOptions)
makePVMcluster(count, ..., options = defaultClusterOptions)
makeMPIcluster(count, ..., options = defaultClusterOptions)
makeNWScluster(names, ..., options = defaultClusterOptions)
getMPIcluster()

Arguments

spec | cluster specification
count | number of nodes to create
names | character vector of node names
options | cluster options object
cl | cluster object
... | cluster option specifications
type | character; specifies cluster type.

Details

makeCluster starts a cluster of the specified or default type and returns a reference to the cluster. Supported cluster types are "SOCK", "PVM", "MPI", and "NWS". For "PVM" and "MPI" clusters the spec argument should be an integer specifying the number of slave nodes to create. For "SOCK" and "NWS" clusters spec should be a character vector naming the hosts on which slave nodes should be started; one node is started for each element in the vector. For "SOCK" and "NWS" clusters spec can also be an integer specifying the number of slaves nodes to create on the local machine.

For SOCK and NWS clusters the spec can also be a list of machine specifications, each a list of named option values. Such a list must include a character value named host host specifying the name or address of the host to use. Any other option can be specified as well. For SOCK and NWS clusters this may be a more convenient alternative than inhomogeneous cluster startup procedure. The options rscript and snowlib are often useful; see the examples below.

stopCluster should be called to properly shut down the cluster before exiting R. If it is not called it may be necessary to use external means to ensure that all slave processes are shut down.

setDefaultClusterOptions can be used to specify alternate values for default cluster options. There are many options. The most useful ones are type and homogeneous. The default value of the type option is currently set to "MPI" if Rmpi is on the search path. Otherwise it is set to "PVM" if the rpvm package is available, to "MPI" if Rmpi is available but rpvm is not, and to "SOCK" if neither of these packages is found.

The homogeneous option should be set to FALSE to specify that the startup procedure for inhomogeneous clusters is to be used; this requires some additional configuration. The default setting is TRUE unless the environment variable R_SNOW_LIB is defined on the master host with a non-empty value.

The optionoutfile can be used to specify the file to which slave node output is to be directed. The default is /dev/null; during debugging of an installation it can be useful to set this to a proper file. On some systems setting outfile to "" or to /dev/tty will result in worker output being sent tothe terminal running the master process.

The functions makeSOCKcluster, makePVMcluster, makeMPIcluster, and makeNWScluster can be used to start a cluster of the corresponding type.

In MPI configurations where process spawning is not available and something like mpirun is used to start a master and a set of slaves the corresponding cluster will have been pre-constructed and can be obtained with getMPIcluster. It is also possible to obtain a reference to the running cluster using makeCluster or makeMPIcluster. In this case the count argument can be omitted; if it is supplied, it must equal the number of nodes in the cluster. This interface is still experimental and subject to change.

For SOCK and NWS clusters the option manual = TRUE forces a manual startup mode in which the master prints the command to be run manually to start a worker process. Together with setting the outfile option this can be useful for debugging cluster startup.

For more details see http://www.stat.uiowa.edu/~luke/R/cluster/cluster.html.

Examples

  ## Not run:
## Two workers run on the local machine as a SOCK cluster.
cl <- makeCluster(c("localhost","localhost"), type = "SOCK")
clusterApply(cl, 1:2, get("+"), 3)
stopCluster(cl)
## Another approach to running on the local machine as a SOCK cluster.
cl <- makeCluster(2, type = "SOCK")
clusterApply(cl, 1:2, get("+"), 3)
stopCluster(cl)
## A SOCK cluster with two workers on Mac OS X, two on Linux, and two
## on Windows:
macOptions <-
    list(host = "owasso",
         rscript = "/Library/Frameworks/R.framework/Resources/bin/Rscript",
         snowlib = "/Library/Frameworks/R.framework/Resources/library/snow")
lnxOptions <-
    list(host = "itasca",
         rscript = "/usr/lib64/R/bin/Rscript",
         snowlib = "/home/luke/tmp/lib")
winOptions <-
    list(host="192.168.1.168",
         rscript="C:/Program Files/R/R-2.7.1/bin/Rscript.exe",
         snowlib="C:/Rlibs")
cl <- makeCluster(c(rep(macoptions, 2), rep(lnxOptions, 2),
  rep(winOptions, 2), type = "SOCK")
clusterApply(cl, 1:2, get("+"), 3)
stopCluster(cl)

## End(Not run)
[Package snow version 0.3-3 Index]