[R-sig-hpc] Rmpi with Open MPI on Debian

[Hao Yu]

Hi Ingeborg,

Dirk already answered some of your questions so I will not repeat here. I
just address some of your issues.

Overall OpenMPI is not a friendly MPI environment to work with unless it
runs under job or resource management such as slurm. This effects how Rmpi
runs under OpenMPI. The way of spawning R slaves under LAM is not working
any more under OpenMPI. Under LAM, one just uses
R -> library(Rmpi) ->  mpi.spawn.Rslaves()
as long as host file is set. Under OpenMPI this leads only one R slave on
the master host no matter how many remote hosts are specified in OpenMPI
hostfile. One has to use orterun to tell Rmpi where remote hosts are.
README in Rmpi shows how to spawn R slaves with mpirun. It applies to
orterun as well. Here is how I do it:
1. Copy R in R's bin to, say, Rort which should be in PATH.
2. Modify Rort to add a line
   R_PROFILE=${R_HOME_DIR}/library/Rmpi/Rprofile; export R_PROFILE
right after R_HOME_DIR
3. To have one master and 4 slaves, run
   orterun -np 5 Rort R CMD BATCH R.in R.out

e.g., I have one R.in like
######
#library(Rmpi)
mpi.universe.size()
#mpi.spawn.Rslaves()
#mpi.setup.rngstream()
mpi.parReplicate(100,mean(rnorm(10000000)))
mpi.close.Rslaves()
mpi.quit()
######
Notice that library(Rmpi) and mpi.spawn.Rslaves are commented out since
Rprofile takes care of loading and spawning.

Regarding 100%cpu usage from slaves while they are waiting, unfortunately,
I cannot fix it until a new release of OpenMPI solves this issue. In Rmpi
0.5-7 I add a few nonblocking parallel apply functions. However they just
reduce cpu usages on master only. I don't know if those slaves will
automatically yield to other programs. Nevertheless, you can try with nice
function. You need to modify Rslaves.sh in Rmpi's inst to add nice in
front of
$R_HOME/bin/R --no-init-file ...

Hao

Ingeborg Schmidt wrote:
> Hello,
> I wish to use Rmpi with Open MPI on Debian. Slaves should be spawned on
> serveral computers which should be able to communicate with a single
> master. However, there does not seem to be a default hostfile for Open MPI
> that is used. So when I use
> library(Rmpi)
> mpi.spawn.Rslaves()
> it only spawns one slaves on the localhost instead of several thereads on
> all my computers. I am unable to find any useful documentation of Open MPI
> (yes, I checked the FAQ on open-mpi.org). Is there such a thing as a
> default hostfile that is used when calling mpi.spawn.Rslaves() ? Or is
> there any other way to use mpi.spawn.Rslaves() with Open MPI so that
> slaves are spawned across multiple computers?
>
> I am unsure about calling R via orterun. The only tutorials regarding
> orterun and R I found (e.g.
> http://dirk.eddelbuettel.com/papers/bocDec2008introHPCwithR.pdf ) seemed
> to imply that there either is no master or the master identifies itself by
> looking at it's mpi.comm.rank() . Moreover running
> paste("I am", mpi.comm.rank(), "of", mpi.comm.size())
> via
> orterun --hostfile MYHOSTFILE -n CPUNUMBER Rslaves.sh RTest.R testlog
> needlog /PATH/TO/R
> results in
> "I am 0 of 0"
> on every node.
> This is not what I want, I would like only the master to execute my R
> script and send relevant methods to the slaves via mpi.bcast.Robj2slave().
> My code contains commands like mpi.remote.exec() which I would like to
> keep. I have not yet seen any examples that are able to combine calling R
> via orterun with communication between the slaves with mpi.remote.exec()
> etc.
>
> By the way: Can you recommend a method to lower the thread priory of the R
> slaves so that other calculations done on the same computers are not
> disturbed? Is placing a nice (Linux command to lower thread priority)
> before R in the Rslaves.sh sufficient when using mpi.spawn.Rslaves()?
>
> Cheers,
> Ingeborg Schmidt
>
>
>
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>


-- 
Department of Statistics & Actuarial Sciences
Fax Phone#:(519)-661-3813
The University of Western Ontario
Office Phone#:(519)-661-3622
London, Ontario N6A 5B7
http://www.stats.uwo.ca/faculty/yu