[R-sig-hpc] Rmpi with Open MPI on Debian

[Dirk Eddelbuettel]

On 11 February 2009 at 14:46, Hao Yu wrote:
| Overall OpenMPI is not a friendly MPI environment to work with unless it
| runs under job or resource management such as slurm. This effects how Rmpi
| runs under OpenMPI. The way of spawning R slaves under LAM is not working
| any more under OpenMPI. Under LAM, one just uses
| R -> library(Rmpi) ->  mpi.spawn.Rslaves()
| as long as host file is set. Under OpenMPI this leads only one R slave on
| the master host no matter how many remote hosts are specified in OpenMPI
| hostfile. One has to use orterun to tell Rmpi where remote hosts are.
| README in Rmpi shows how to spawn R slaves with mpirun. It applies to
| orterun as well. Here is how I do it:
| 1. Copy R in R's bin to, say, Rort which should be in PATH.
| 2. Modify Rort to add a line
|    R_PROFILE=${R_HOME_DIR}/library/Rmpi/Rprofile; export R_PROFILE
| right after R_HOME_DIR
| 3. To have one master and 4 slaves, run
|    orterun -np 5 Rort R CMD BATCH R.in R.out
| 
| e.g., I have one R.in like
| ######
| #library(Rmpi)
| mpi.universe.size()
| #mpi.spawn.Rslaves()
| #mpi.setup.rngstream()
| mpi.parReplicate(100,mean(rnorm(10000000)))
| mpi.close.Rslaves()
| mpi.quit()
| ######
| Notice that library(Rmpi) and mpi.spawn.Rslaves are commented out since
| Rprofile takes care of loading and spawning.

Very interesting. You can do the same with littler as well, defining
the R_PROFILE environment variable on the fly in the shell:

edd at ron:~/src/progs/R> R_PROFILE=/usr/lib/R/site-library/Rmpi/Rprofile orterun -n 3 ./rmpitest.r
master (rank 0, comm 1) of size 3 is running on: ron
slave1 (rank 1, comm 1) of size 3 is running on: ron
slave2 (rank 2, comm 1) of size 3 is running on: ron
Loading required package: rlecuyer
Loading required package: rlecuyer
Loading required package: rlecuyer
      Min.    1st Qu.     Median       Mean    3rd Qu.       Max.
-0.0012100 -0.0003179  0.0001109  0.0002848  0.0007210  0.0032570
edd at ron:~/src/progs/R>


where I used 


edd at ron:~/src/progs/R> cat rmpitest.r
#!/usr/bin/r

# Start via
#   R_PROFILE=/usr/lib/R/site-library/Rmpi/Rprofile orterun -n 3 ./rmpitest.r
# which loads Rmpi and spawns

mpi.universe.size()
mpi.setup.rngstream()
res <- mpi.parReplicate(20,mean(rnorm(1000000)))   # reduced n and N
mpi.close.Rslaves()
print(summary(res))
mpi.quit()


Dirk


-- 
Three out of two people have difficulties with fractions.