[R] Multicore computation in Windows network: How to set up Rmpi

[Samu Mäntyniemi]

Hello!

I finally got MPICH 1.06 + R 2.6.1 + Rmpi 0.5-5 working with multiple 
computers. The key was to realize that the number of processes should be 
one when launching Rgui using mpiexec and not the number of 
master+slaves, as I had first wrongly understood.

However, I seem to have a new problem which I have not been able to 
figure out:

After loading Rmpi, the first attempt to mpi.spawn.Rslaves() always 
spawns the slaves on the local machine instead of on both machines. If I 
close the slaves and spawn again, then one slave gets spawned on remote 
machine. Each time I close and then spawn againg, the order of machines 
is different, and eventually I get back to the situation where all 
slaves are on the local machine. Continuing to do spawning and closing 
seems to reveal a pattern. I can see similar behavior if I have more 
than two machines, and it takes more spawn-close cycles to get all my 
slave machines spawned on.

Below is an example session with two machines. This pattern shows 
everytime I start R and run this script. How to control the spawning so 
that I get everything right at the first call of mpi.spawn.Rslaves()?

Regards,

Samu

<R>

 >
 > library(Rmpi)
 > sessionInfo()
R version 2.6.1 (2007-11-26)
i386-pc-mingw32

locale:
LC_COLLATE=Finnish_Finland.1252;LC_CTYPE=Finnish_Finland.1252;LC_MONETARY=Finnish_Finland.1252;LC_NUMERIC=C;LC_TIME=Finnish_Finland.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] Rmpi_0.5-5
 > mpi.universe.size()
[1] 2
 > mpichhosts()
          master          slave1          slave2
"clustermaster" "clustermaster" "clusterslave1"
 > mpi.spawn.Rslaves()
         2 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 3 is running on: ClusterMaster
slave1 (rank 1, comm 1) of size 3 is running on: ClusterMaster
slave2 (rank 2, comm 1) of size 3 is running on: ClusterMaster
 > mpi.close.Rslaves()
[1] 1
 > mpi.spawn.Rslaves()
         2 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 3 is running on: ClusterMaster
slave1 (rank 1, comm 1) of size 3 is running on: ClusterSlave1
slave2 (rank 2, comm 1) of size 3 is running on: ClusterMaster
 > mpi.close.Rslaves()
[1] 1
 > mpi.spawn.Rslaves()
         2 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 3 is running on: ClusterMaster
slave1 (rank 1, comm 1) of size 3 is running on: ClusterMaster
slave2 (rank 2, comm 1) of size 3 is running on: ClusterSlave1
 > mpi.close.Rslaves()
[1] 1
 > mpi.spawn.Rslaves()
         2 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 3 is running on: ClusterMaster
slave1 (rank 1, comm 1) of size 3 is running on: ClusterMaster
slave2 (rank 2, comm 1) of size 3 is running on: ClusterMaster
 > mpi.close.Rslaves()
[1] 1
 > mpi.spawn.Rslaves()
         2 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 3 is running on: ClusterMaster
slave1 (rank 1, comm 1) of size 3 is running on: ClusterSlave1
slave2 (rank 2, comm 1) of size 3 is running on: ClusterMaster
 > mpi.close.Rslaves()
[1] 1
 > mpi.spawn.Rslaves()
         2 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 3 is running on: ClusterMaster
slave1 (rank 1, comm 1) of size 3 is running on: ClusterMaster
slave2 (rank 2, comm 1) of size 3 is running on: ClusterSlave1
 > mpi.close.Rslaves()
[1] 1
 >
 >
 > mpi.spawn.Rslaves()
         2 slaves are spawned successfully. 0 failed.
master (rank 0, comm 1) of size 3 is running on: ClusterMaster
slave1 (rank 1, comm 1) of size 3 is running on: ClusterMaster
slave2 (rank 2, comm 1) of size 3 is running on: ClusterMaster
 > mpi.close.Rslaves()
[1] 1
 >

</R>


Samu Mäntyniemi kirjoitti:
> Some progress in my problem:
> 
> Samu Mäntyniemi kirjoitti:
> 
>> With MPICH2 I managed to connect my computers so that I was able to
>> remotely launch Rgui on both machines but R hanged when calling
>> "library(Rmpi)". If only one Rgui was launched on the localhost,
>> "library(Rmpi)" worked without errors, but trying to use
>> "mpi.spawn.Rslaves()" resulted in an error message, and so did
>> "mpi.universe.size()". (In my current setup I can not reproduce this 
>> error message, but I can go back to this setup if this seems to be an 
>> important piece of information)
> 
> I vent back to MPICH2 installation to see what the error was:
> "ERROR in names(HOSTNAMES)<-base: attempt to set an attribute on NULL"
> 
> Trying to rethink what the problem was I realized that unlike in 
> DeinoMPI, I need to write the host names manually on the "configurable 
> settings" -window, and in order to have one cpu available on the local 
> machine, I need to write "myhostname:2".
> 
> After these changes MPICH2 1.06 +R-2.6.0+Rmpi 0.5-5 work on the single 
> machine in the same way as my DeinoMPI installation: Correct number of 
> cpu:s is detected and I can "mpi.spawn.Rslaves()"
> 
> I will try to do this with two hosts next and see if there is more luck 
> with MPICH2 than DeinoMPI.
> 
> Samu
> 
> 
> 
> 
> ------------------------------------------
> Samu Mäntyniemi
> Researcher
> Fisheries and Environmental Management Group (FEM)
> Department of Biological and Environmental Sciences
> Biocenter 3, room 4414
> Viikinkaari 1
> P.O. Box 65
> FIN-00014 University of Helsinki
> 
> Phone: +358 9 191 58710
> Fax: +358 9 191 58257
> 
> email: samu.mantyniemi at helsinki.fi
> personal webpage: http://www.helsinki.fi/people/samu.mantyniemi/
> FEM webpage: http://www.helsinki.fi/science/fem/
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 


-- 
------------------------------------------
Samu Mäntyniemi
Researcher
Fisheries and Environmental Management Group (FEM)
Department of Biological and Environmental Sciences
Biocenter 3, room 4414
Viikinkaari 1
P.O. Box 65
FIN-00014 University of Helsinki

Phone: +358 9 191 58710
Fax: +358 9 191 58257

email: samu.mantyniemi at helsinki.fi
personal webpage: http://www.helsinki.fi/people/samu.mantyniemi/
FEM webpage: http://www.helsinki.fi/science/fem/