[R-sig-hpc] Rmpi spawning across nodes.

[Brian G. Peterson]

On Thu, 2012-03-29 at 14:26 -0400, Ben Weinstein wrote:
> Hello all,
> 
> I've seen multiple posts on this subject, but haven't clearly been able to
> understand the issue. There must be something small.
> 
> I am trying to get Rmpi, foreach and snow working on a beowulf cluster,
> using debian.
> 
> I have succeeded in installing Rmpi, changing the paths, etc.
> 
> However, i remain the with my original problem, when
> 
> cl<-makeCluster(4,type="MPI")
> 
> spawns slaves, they are always on the same node!

<... snip ...>

> My dream scenerio is to use both processors on each node, and pass
> information between nodes in separate workflows.
> 
> I appreciate any suggestions
> 
> There is nothing wrong with my R script in terms of code, it works great
> using doMC and foreach, with 8 cores on my desktop. However, i have not
> been able to register the correct cores when using the University Cluster.
> 
> I am new at this, please forgive my lack of precise terms.

I'll suggest a few things.

First, if you want to use MPI, doMPI is less fragile than doSNOW for an
MPI cluster.

Second, I recall that there was a config file that needed to be set to
define the worker machines for our MPI cluster (I don't use MPI
routinely anymore).  This had some interaction that I don't recall in
detail that should be described in the doMPI vignette.

Third, if you want to use foreach and parallelization in an interactive
session, as you can with doMC or doParallel, I recommend looking into
doRedis.

Regards,

   - Brian


-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock