[R] Mpirun with R CMD scripts

[Martin Morgan]

Your description was a bit hard for me to follow. I think what you're
trying to do is something like

TestRmpi.R
----------
library(Rmpi)
if (0==mpi.comm.rank(comm=0)) {
  # 'manager', e.g., cat("manager\n")
} else {
  # 'worker', e.g., cat(mpi.comm.rank(comm=0),"worker\n")
}
mpi.quit()

TestRmpi.sh
-----------
R --slave < $1

and from the command line

mpirun -np 3 TestRmpi.sh TestRmpi.R

If instead you try

mpirun -np 3 R CMD BATCH TestRmpi.R

then each node executes R CMD BATCH. As a consequence, each node
writes to a file TestRmpi.Rout, and you end up with an undetermined
outcome (because the different nodes are all trying to write to the
same file).

If your file were

WRONGTestSnow.R
---------------
library(snow)
cl <- makeCluster(3,type="MPI")
clusterCall(cl,function() Sys.info()[c("nodename","machine")])
stopCluster(cl)

then you would execute makeCluster on each node, for a total of 9
instances of R. Probably not what you want.

I found that snow was useful for interactive and exploratory work, but
that Rmpi (or rpvm) provide the level of control that you really need
for more substantial use of parallel resources. Recent versions of
Rmpi have mpi.parLapply and related, which are like the high-level
functions in snow.

Hope that helps,

Martin

"Srividya Valivarthi" <srividya.v at gmail.com> writes:

> Hi,
>
>    I am working on a 64-bit rocks cluster and am relatively new to the
> R package. I am trying to get Snow working with R and Rmpi and have
> run into the following issue. R is able to load the Rmpi and snow
> libraries and is able to run simple commands both interactively and
> batch as follows:
>
> -------------------------------------------------------------------------------------------------------
> [srividya at cheaha ~]$ R
> R : Copyright 2005, The R Foundation for Statistical Computing
> Version 2.2.0  (2005-10-06 r35749)
> ISBN 3-900051-07-0
>> library(snow)
>> c1<-makeCluster(3, type="MPI")
> Loading required package: Rmpi
>         3 slaves are spawned successfully. 0 failed.
>>
>>
>>
>> clusterCall(c1,function() Sys.info()[c("nodename","machine")])
> [[1]]
>            nodename             machine
> "cheaha.ac.uab.edu"            "x86_64"
>
> [[2]]
>             nodename              machine
> "compute-0-12.local"             "x86_64"
>
> [[3]]
>             nodename              machine
> "compute-0-13.local"             "x86_64"
>
>> stopCluster(c1)
> [1] 1
>> q()
> --------------------------------------
>
> But on running the same script with mpirun i get the following error.
> *****************************************************************8
> [srividya at cheaha ~]$ mpirun -np 3 R -slave R CMD BATCH TestSnow.R
> /home/srividya/R/library/snow/RMPInode.sh: line 9: 19431 Segmentation
> fault      ${RPROG:-R} --vanilla  >${OUT:-/dev/null} 2>&1 <<EOF
>
> library(Rmpi)
> library(snow)
>
> runMPIslave()
> EOF
> **************************************
>
> I am not sure about what the error could be and any help is greatly
> appreaciated.
>
> Thanks,
> Srividya
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html