[R-sig-hpc] Error in makeMPIcluster(spec, ...): how to get a minimal example for parallel computing with doSNOW to run?

[Marius Hofert]

Dear expeRts,

I try to get a minimal example for parallel computing via "foreach" + "doSNOW" to run on a computer cluster (Brutus from ETH Zurich). The minimal example is given below. It runs perfectly fine on my MacBook but when I submit it as a batch job via ...
bsub -n 3 -R "select[model==Opteron8380]" mpirun R --no-save -q -f doSNOW_minimal.R
... it does not work. The output is also given below. The error is "Error in makeMPIcluster(spec, ...) : a cluster already exists 1". The only similar thing I found on the web is http://www.mail-archive.com/r-help@stat.math.ethz.ch/msg35501.html
but unfortunately, it was not answered. 
I also contacted the maintainers of the cluster, however, they couldn't really help me since they were no "expeRts". 

Cheers,

Marius

Ps: As you an expeRt probably sees right away, I am a newby when it comes to parallel computing, so please check all the details (does the mpirun command makes sense? and the bsubs command [the batch system is "LSF"]?) and let me know if you need further. 

## ==== minimal example ==== 

library(doSNOW) # loads foreach
library(Rmpi) # for default in makeCluster()
cl <- makeCluster(3) # create cluster object with the given number of slaves 
registerDoSNOW(cl) # register the cluster object with foreach
x <- foreach(i = 1:3) %dopar% { # simple test
   sqrt(i)
}
x 
stopCluster(cl) # properly shut down the cluster 

## ==== minimal example ====

## ==== output ====

Sender: LSF System <lsfadmin at a6218>
Subject: Job 921476: <mpirun R --no-save -q -f doSNOW_minimal.R> Done

Job <mpirun R --no-save -q -f doSNOW_minimal.R> was submitted from host <brutus2> by user <hofertj> in cluster <brutus>.
Job was executed on host(s) <3*a6218>, in queue <pub.1h>, as user <hofertj> in cluster <brutus>.
</cluster/home/math/hofertj> was used as the home directory.
</cluster/home/math/hofertj> was used as the working directory.
Started at Thu Dec 16 22:16:31 2010
Results reported at Thu Dec 16 22:16:43 2010

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
mpirun R --no-save -q -f doSNOW_minimal.R
------------------------------------------------------------

Successfully completed.

Resource usage summary:

    CPU time   :      6.96 sec.
    Max Memory :         3 MB
    Max Swap   :        29 MB

    Max Processes  :         1
    Max Threads    :         1

The output (if any) follows:

master (rank 0, comm 1) of size 3 is running on: a6218 
slave1 (rank 1, comm 1) of size 3 is running on: a6218 
slave2 (rank 2, comm 1) of size 3 is running on: a6218 
> library(doSNOW) # loads foreach
Loading required package: foreach
Loading required package: iterators
Loading required package: codetools
Loading required package: snow
> library(Rmpi) # for default in makeCluster()
> cl <- makeCluster(3) # create cluster object with the given number of slaves 
Error in makeMPIcluster(spec, ...) : a cluster already exists 1
Calls: makeCluster -> makeMPIcluster
> registerDoSNOW(cl) # register the cluster object with foreach
Error in assign("data", data, pos = .foreachGlobals, inherits = FALSE) : 
  object 'cl' not found
Calls: registerDoSNOW -> setDoPar -> assign
> x <- foreach(i = 1:3) %dopar% { # simple test
+    sqrt(i)
+ }
Error in checkCluster(cl) : not a valid cluster
Calls: %dopar% -> <Anonymous> -> clusterCall -> checkCluster
> x 
Error: object 'x' not found
> stopCluster(cl) # properly shut down the cluster 
Error in stopCluster(cl) : object 'cl' not found
> 
> 
[1] "Please use mpi.close.Rslaves() to close slaves"
[1] "Please use mpi.quit() to quit R"

## ==== output ====