[R] snow/Rmpi without MPI.spawn?

[Martin Morgan]

On 09/03/2014 10:24 PM, Leek, Jim wrote:
> Thanks for the tips.  I'll take a look around for for loops in the morning.
>
> I think the example you provided worked for OpenMPI.  (The default on our machine is MPICH2, but it gave the same error about calling spawn.)  Anyway, with OpenMPI I got this:
>
>> # salloc -n 12 orterun -n 1 R -f spawn.R
>> library(Rmpi)
>> ## Recent Rmpi bug -- should be mpi.universe.size() nWorkers <- mpi.universe.size()

(the '## Recent Rmpi bug' comment should have been removed, it's a holdover from 
when the script was written several years ago)

>> nslaves = 4
>> mpi.spawn.Rslaves(nslaves)

The argument needs to be named

   mpi.spawn.Rslaves(nslaves=4)

otherwise R matches unnamed arguments by position, and '4' is associated with 
the 'Rscript' argument.

Martin

> Reported: 2 (out of 2) daemons - 4 (out of 4) procs
>
> Then it hung there.  So things spawned anyway, which is progress.  I'm just not sure is that expected behavior for parSupply or not.
>
> Jim
>
> -----Original Message-----
> From: Martin Morgan [mailto:mtmorgan at fhcrc.org]
> Sent: Wednesday, September 03, 2014 5:08 PM
> To: Leek, Jim; r-help at r-project.org
> Subject: Re: [R] snow/Rmpi without MPI.spawn?
>
> On 09/03/2014 03:25 PM, Jim Leek wrote:
>> I'm a programmer at a high-performance computing center.  I'm not very
>> familiar with R, but I have used MPI from C, C++, and Python.  I have
>> to run an R code provided by a guy who knows R, but not MPI.  So, this
>> fellow used the R snow library to parallelize his R code
>> (theoretically, I'm not actually sure what he did.)  I need to get
>> this code running on our machines.
>>
>> However, Rmpi and snow seem to require mpi spawn, which our computing
>> center doesn't support.  I even tried building Rmpi with MPICH1
>> instead of 2, because Rmpi has that option, but it still tries to use spawn.
>>
>> I can launch plenty of processes, but I have to launch them all at
>> once at the beginning. Is there any way to convince Rmpi to just use
>> those processes rather than trying to spawn its own?  I haven't found
>> any documentation on this issue, although I would've thought it would be quite common.
>
> This script
>
> spawn.R
> =======
> # salloc -n 12 orterun -n 1 R -f spawn.R
> library(Rmpi)
> ## Recent Rmpi bug -- should be mpi.universe.size() nWorkers <- mpi.universe.size()
> mpi.spawn.Rslaves(nslaves=nWorkers)
> mpiRank <- function(i)
>     c(i=i, rank=mpi.comm.rank())
> mpi.parSapply(seq_len(2*nWorkers), mpiRank)
> mpi.close.Rslaves()
> mpi.quit()
>
> can be run like the comment suggests
>
>      salloc -n 12 orterun -n 1 R -f spawn.R
>
> uses slurm (or whatever job manager) to allocate resources for 12 tasks and spawn within that allocation. Maybe that's 'good enough' -- spawning within the assigned allocation? Likely this requires minimal modification of the current code.
>
> More extensive is to revise the manager/worker-style code to something more like single instruction, multiple data
>
>
> simd.R
> ======
> ## salloc -n 4 orterun R --slave -f simd.R
> sink("/dev/null") # don't capture output -- more care needed here
> library(Rmpi)
>
> TAGS = list(FROM_WORKER=1L)
> .comm = 0L
>
> ## shared `work', here just determine rank and host
> work = c(rank=mpi.comm.rank(.comm),
>            host=system("hostname", intern=TRUE))
>
> if (mpi.comm.rank(.comm) == 0) {
>       ## manager
>       mpi.barrier(.comm)
>       nWorkers = mpi.comm.size(.comm)
>       res = list(nWorkers)
>       for (i in seq_len(nWorkers - 1L)) {
>           res[[i]] <- mpi.recv.Robj(mpi.any.source(), TAGS$FROM_WORKER,
>                                     comm=.comm)
>       }
>       res[[nWorkers]] = work
>       sink() # start capturing output
>       print(do.call(rbind, res))
> } else {
>       ## worker
>       mpi.barrier(.comm)
>       mpi.send.Robj(work, 0L, TAGS$FROM_WORKER, comm=.comm)
> }
> mpi.quit()
>
> but this likely requires some serious code revision; if going this route then
> http://r-pbd.org/ might be helpful (and from a similar HPC environment).
>
> It's always worth asking whether the code is written to be efficient in R -- a
> typical 'mistake' is to write R-level explicit 'for' loops that
> "copy-and-append" results, along the lines of
>
>      len <- 100000
>      result <- NULL
>      for (i in seq_len(len))
>          ## some complicated calculation, then...
>          result <- c(result, sqrt(i))
>
> whereas it's much better to "pre-allocate and fill"
>
>       result <- integer(len)
>       for (i in seq_len(len))
>           result[[i]] = sqrt(i)
>
> or
>
>       lapply(seq_len(len), sqrt)
>
> and very much better still to 'vectorize'
>
>       result <- sqrt(seq_len(len))
>
> (timing for me are about 1 minute for "copy-and-append", .2 s for "pre-allocate
> and fill", and .002s for "vectorize").
>
> Pushing back on the guy providing the code (grep for "for" loops, and look for
> that copy-and-append pattern) might save you from having to use parallel
> evaluation at all.
>
> Martin
>
>>
>> Thanks,
>> Jim
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>


-- 
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793