[R-sig-hpc] Setting up Rmpi, snow, snowfall on hpc

[Kevin.Buckley at ecs.vuw.ac.nz]

> Sorry, if this appears to be a naive question! I was trying
> to set up a parallel R working environment on my account on
> hpc (running SGE). I tried to install Rmpi, but wasn't successful.
> What do I need to get this going ?

Hey Brian,

a few things around and about this:

In terms of building the package, look for a thread entitled:

 "Trouble using Rmpi after successful installation"

from April last year (2010), and one entitled

  "trouble installing Rmpi"

from November last year (2010), which contain some info about
the way I got similar stuff going, especially in directing the
R build process to use the OpenMPI compiler wrappers.


And as someone else has already suggested: LAM has been pretty much
absorbed into OpenMPI.


Within an SGE/OpenMPI environment, I got things going by taking
a wrapper from the RmpiSNOW package and tailoring that for
invoking plain Rmpi scripts.

This allowed an SGE submission, into an SGE parallel environment,
to contain a line akin to:

mpirun -np $NSLOTS /usr/local/bin/RMPI -f /some/path/to/an/Rmpi_script.R
--args 2 5

where that "/usr/local/bin/RMPI" wrapper sets up the various paths
and, eg, Rlibs env var, for your actual script, whilst "$NSLOTS" gets
set by the SGE when the parallel queue gets requested, eg.

 qsub -pe pe_mpi 2 qsub_script.sh

This has even worked for Rmpi scripts submitted via a Globus
grid gateway into an SGE grid.

Kevin

PS
You'll find that the RmpiSNOW packages script has a stanza
that mentions ${LAMRANK} and "LAM-MPI's mpirun", but ignore
it and go with OpenMPI!

-- 
Kevin M. Buckley                                  Room:  CO327
School of Engineering and                         Phone: +64 4 463 5971
 Computer Science
Victoria University of Wellington
New Zealand