[R-sig-hpc] parallel() requires snow package to create an MPI cluster?

Bennet Fauber bennet at umich.edu
Sat Nov 8 20:11:24 CET 2014


I installed from source R 3.1.1 and Rmpi 0.6-5.  Both pass tests, and
both work.  In the past, I have always also installed multicore and
snow, but I was under the impression that parallel superceded those
two packages.  That seems to be at least partially incorrect...?

If I want to use the functions from parallel() to create a cluster
object using MPI, is snow a required package?  I was getting namespace
errors while loading functions for snow when I tried it without snow
installed, and after installing snow, it all just seems to work.

Is snow required for

cl <- makeCluster(mpi.universe.size()-1, "MPI")

to work?  I start R from an interactive PBS session with two nodes and
four processors using

$ mpirun -np 1 R --no-save --no-restore

Then I get,

> mpi.universe.size()
mpi.universe.size()
[1] 4

> sessionInfo()
sessionInfo()
R version 3.1.1 (2014-07-10)
Platform: x86_64-unknown-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] graphics  grDevices datasets  stats     utils     methods   base

other attached packages:
[1] Rmpi_0.6-5


> library(parallel)
library(parallel)

> cl <- makeCluster(mpi.universe.size()-1, "MPI")
cl <- makeCluster(mpi.universe.size()-1, "MPI")
Error in loadNamespace(name) : there is no package called ‘snow’
Calls: makeCluster ... tryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>

With snow in the library path, I instead get as the response to the
last command,

> cl <- makeCluster(mpi.universe.size()-1, "MPI")
cl <- makeCluster(mpi.universe.size()-1, "MPI")
    3 slaves are spawned successfully. 0 failed.


Is that correct and normal?  For MPI clusters, both Rmpi and snow are
rerequisites to prallel()'s makeCluster(..., "MPI")?  I was kind of
under the impression that parallel() was self-contained and superceded
both multicore and snow.

Or did I manage to somehow misconfigure or misinstall things?

Thanks,  -- bennet



More information about the R-sig-hpc mailing list