[R] Multicore computation in Windows network: How to set up Rmpi

[Samu Mäntyniemi]

R-users,

My question is related to earlier posts about benefits of quadcore over
dualcore computers; I am trying to setup a cluster of windows xp
computers so that eventually I could make use of 10-20 cpu:s, but for
learning how to do this, I am playing around with two laptops.

I thought that the package snow would come handy in this situation, but
to use snow, I would probably need to install package Rmpi first. I
might also like to use Rmpi directly.

In order to use Rmpi in windows, I need to install MPI middle-ware, for
which I have found two options:

MPICH2: http://www.mcs.anl.gov/research/projects/mpich2/

DeinoMPI: http://mpi.deino.net/

First I tried MPICH2 1.06 + R-2.6.0 + Rmpi 0.5-5. (I downloaded windows
binaries of Rmpi from Rmpi website
:http://www.stats.uwo.ca/faculty/yu/Rmpi/)

With MPICH2 I managed to connect my computers so that I was able to
remotely launch Rgui on both machines but R hanged when calling
"library(Rmpi)". If only one Rgui was launched on the localhost,
"library(Rmpi)" worked without errors, but trying to use
"mpi.spawn.Rslaves()" resulted in an error message, and so did
"mpi.universe.size()". (In my current setup I can not reproduce this 
error message, but I can go back to this setup if this seems to be an 
important piece of information)

After that I removed MPICH2 from the system and installed DeinoMPI
instead, thus my setup was DeinoMPI 1.1.0 + R-2.6.0 + Rmpi 0.5-6.
Using this setup on a single machine seems to work,
"mpi.universe.size()" returns the correct number of cpu cores and
"mpi.spawn.Rslaves()" creates one master and one slave on the local 
unicore computer. However, trying to use two computers results in 
similar behavior as with MPICH2: Rgui gets started on both, but R hangs 
when trying "library(Rmpi)".

Following the advice given at the Rmpi website, this behavior could be
due to firewall settings. However, the result is the same if I take down
all my firewalls. Trying to debug this, I tried to run the example MPI
programs provided with the DeinoMPI installation. The example programs
work as expected: both machines participate to parallel computation and
the result is shown on the master node. This makes me believe that the
problem is likely related to R and Rmpi configuration, or the settings
I used when launching Rgui using mpiexec:

Settings on Mpiexec- tab:
Application: "C:\Program Files\R\R-2.6.0\bin\Rgui.exe"
Number of processes: 2
hosts: "akva26 samu" These are the computers with DeinoMPI installed.
DeinoMPI cluster-tab shows that they are ready to accept MPI jobs.)
localroot: checked

Other options were just empty.

I am sure that someone has tried these before, and I was hoping to find 
such users from this mailing list. Could you kindly share your 
experiences about this issue?

For example, does anyone have a working setup with DeinoMPI? According 
to Rmpi website DeinoMPI is the easiest way to set up MPI for a single 
windows machine, but I am not sure whether this is also intended to mean 
that one can not expect it to work with multiple computers.

Regards,

Samu Mäntyniemi


------------------------------------------
Samu Mäntyniemi
Researcher
Fisheries and Environmental Management Group (FEM)
Department of Biological and Environmental Sciences
Biocenter 3, room 4414
Viikinkaari 1
P.O. Box 65
FIN-00014 University of Helsinki

Phone: +358 9 191 58710
Fax: +358 9 191 58257

email: samu.mantyniemi  helsinki.fi
personal webpage: http://www.helsinki.fi/people/samu.mantyniemi/
FEM webpage: http://www.helsinki.fi/science/fem/