[R-sig-hpc] Setting up Rmpi, snow, snowfall on hpc
Kevin.Buckley at ecs.vuw.ac.nz
Kevin.Buckley at ecs.vuw.ac.nz
Wed Mar 16 00:30:21 CET 2011
> Sorry, if this appears to be a naive question! I was trying
> to set up a parallel R working environment on my account on
> hpc (running SGE). I tried to install Rmpi, but wasn't successful.
> What do I need to get this going ?
Hey Brian,
a few things around and about this:
In terms of building the package, look for a thread entitled:
"Trouble using Rmpi after successful installation"
from April last year (2010), and one entitled
"trouble installing Rmpi"
from November last year (2010), which contain some info about
the way I got similar stuff going, especially in directing the
R build process to use the OpenMPI compiler wrappers.
And as someone else has already suggested: LAM has been pretty much
absorbed into OpenMPI.
Within an SGE/OpenMPI environment, I got things going by taking
a wrapper from the RmpiSNOW package and tailoring that for
invoking plain Rmpi scripts.
This allowed an SGE submission, into an SGE parallel environment,
to contain a line akin to:
mpirun -np $NSLOTS /usr/local/bin/RMPI -f /some/path/to/an/Rmpi_script.R
--args 2 5
where that "/usr/local/bin/RMPI" wrapper sets up the various paths
and, eg, Rlibs env var, for your actual script, whilst "$NSLOTS" gets
set by the SGE when the parallel queue gets requested, eg.
qsub -pe pe_mpi 2 qsub_script.sh
This has even worked for Rmpi scripts submitted via a Globus
grid gateway into an SGE grid.
Kevin
PS
You'll find that the RmpiSNOW packages script has a stanza
that mentions ${LAMRANK} and "LAM-MPI's mpirun", but ignore
it and go with OpenMPI!
--
Kevin M. Buckley Room: CO327
School of Engineering and Phone: +64 4 463 5971
Computer Science
Victoria University of Wellington
New Zealand
More information about the R-sig-hpc
mailing list