[R] [R-sig-hpc] Error in makeMPIcluster(spec, ...): how to get a minimal example for parallel computing with doSNOW to run?

[Mario Valle]

Shouldn't -n be 4 in the bsub command?
One master+3 slaves. This was required for snowfall, but I think doSNOW 
is similar.
Hope it helps
                                         mario

On 16-Dec-10 23:09, Marius Hofert wrote:
> Dear expeRts,
>
> I try to get a minimal example for parallel computing via "foreach" + "doSNOW" to run on a computer cluster (Brutus from ETH Zurich). The minimal example is given below. It runs perfectly fine on my MacBook but when I submit it as a batch job via ...
> bsub -n 3 -R "select[model==Opteron8380]" mpirun R --no-save -q -f doSNOW_minimal.R
> ... it does not work. The output is also given below. The error is "Error in makeMPIcluster(spec, ...) : a cluster already exists 1". The only similar thing I found on the web is http://www.mail-archive.com/r-help@stat.math.ethz.ch/msg35501.html
> but unfortunately, it was not answered.
> I also contacted the maintainers of the cluster, however, they couldn't really help me since they were no "expeRts".
>
> Cheers,
>
> Marius
>
> Ps: As you an expeRt probably sees right away, I am a newby when it comes to parallel computing, so please check all the details (does the mpirun command makes sense? and the bsubs command [the batch system is "LSF"]?) and let me know if you need further.
>
> ## ==== minimal example ====
>
> library(doSNOW) # loads foreach
> library(Rmpi) # for default in makeCluster()
> cl<- makeCluster(3) # create cluster object with the given number of slaves
> registerDoSNOW(cl) # register the cluster object with foreach
> x<- foreach(i = 1:3) %dopar% { # simple test
>     sqrt(i)
> }
> x
> stopCluster(cl) # properly shut down the cluster
>
> ## ==== minimal example ====
>
> ## ==== output ====
>
> Sender: LSF System<lsfadmin at a6218>
> Subject: Job 921476:<mpirun R --no-save -q -f doSNOW_minimal.R>  Done
>
> Job<mpirun R --no-save -q -f doSNOW_minimal.R>  was submitted from host<brutus2>  by user<hofertj>  in cluster<brutus>.
> Job was executed on host(s)<3*a6218>, in queue<pub.1h>, as user<hofertj>  in cluster<brutus>.
> </cluster/home/math/hofertj>  was used as the home directory.
> </cluster/home/math/hofertj>  was used as the working directory.
> Started at Thu Dec 16 22:16:31 2010
> Results reported at Thu Dec 16 22:16:43 2010
>
> Your job looked like:
>
> ------------------------------------------------------------
> # LSBATCH: User input
> mpirun R --no-save -q -f doSNOW_minimal.R
> ------------------------------------------------------------
>
> Successfully completed.
>
> Resource usage summary:
>
>      CPU time   :      6.96 sec.
>      Max Memory :         3 MB
>      Max Swap   :        29 MB
>
>      Max Processes  :         1
>      Max Threads    :         1
>
> The output (if any) follows:
>
> master (rank 0, comm 1) of size 3 is running on: a6218
> slave1 (rank 1, comm 1) of size 3 is running on: a6218
> slave2 (rank 2, comm 1) of size 3 is running on: a6218
>> library(doSNOW) # loads foreach
> Loading required package: foreach
> Loading required package: iterators
> Loading required package: codetools
> Loading required package: snow
>> library(Rmpi) # for default in makeCluster()
>> cl<- makeCluster(3) # create cluster object with the given number of slaves
> Error in makeMPIcluster(spec, ...) : a cluster already exists 1
> Calls: makeCluster ->  makeMPIcluster
>> registerDoSNOW(cl) # register the cluster object with foreach
> Error in assign("data", data, pos = .foreachGlobals, inherits = FALSE) :
>    object 'cl' not found
> Calls: registerDoSNOW ->  setDoPar ->  assign
>> x<- foreach(i = 1:3) %dopar% { # simple test
> +    sqrt(i)
> + }
> Error in checkCluster(cl) : not a valid cluster
> Calls: %dopar% ->  <Anonymous>  ->  clusterCall ->  checkCluster
>> x
> Error: object 'x' not found
>> stopCluster(cl) # properly shut down the cluster
> Error in stopCluster(cl) : object 'cl' not found
>>
> [1] "Please use mpi.close.Rslaves() to close slaves"
> [1] "Please use mpi.quit() to quit R"
>
> ## ==== output ====
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc

-- 
Ing. Mario Valle
Data Analysis and Visualization Group            | http://www.cscs.ch/~mvalle
Swiss National Supercomputing Centre (CSCS)      | Tel:  +41 (91) 610.82.60
v. Cantonale Galleria 2, 6928 Manno, Switzerland | Fax:  +41 (91) 610.82.82