[R-sig-hpc] SNOW Hybrid Cluster in R, Network problems
Martin Seilmayer
M.Seilmayer at hzdr.de
Tue Jul 3 17:09:09 CEST 2012
Hi Steve,
:) so you pointed out the most interesting problem in this challenge:
"How to start (fast) a remote process on Windows from a console"
The aim was to find a clean solution without mixing up Windows with
Linux staff like ssh or PsTools or rsh or vis versa. With MPICH2, smpd
service/daemon running, it is possible so start processes on each worker.
I am working on a Windows machine, so the R command to start a
Windows-worker is as follows:
system(paste("mpiexec -hosts ",length(nodestostart),"
",paste(nodestostart,collapse=" ")," C:\\MPI_R_Start.bat
",hostname,sep=""),wait=F)
"wait = F" important, because the next command should be
makeCluster(listofnodes,type=SOCK)
So I wrote a batch-file for windows and a shell script for Linux, which
sets up each machine individually and which is started remote form
MPICH2. This config script starts as many Rscripts as I had configured
before. The Master on the other side knows how many workers are
available an each machine.
Important! It takes no configuration like rshcmd (because ssh is not
available on Win). One must guarantee that "makeCluster()" comes after
"starting Workers" thats the "trick". This works fine, because
makeCluster() makes the master waiting for each worker calling back. If
the master is not responding, the worker shuts down, so far in the Linux
world.
And finally: Yes Mpich2 is able to start every program / command on a
remote machine. Thats a bit of a security problem, if you were very
strong and restrictive to me.
I hope I answered in your sense!
Martin
Dipl.-Ing. Martin Seilmayer
Helmholtz-Zentrum Dresden-Rossendorf e. V.
Institut fuer Fluiddynamik
Abteilung Magnetohydrodynamik
Bautzner Landstraße 400
01328 Dresden, Germany
@fon: +49 351 260 3165
@fax: +49 351 260 12969
@web: www.hzdr.de
Am 03.07.2012 16:44, schrieb Stephen Weston:
> On Tue, Jul 3, 2012 at 5:31 AM, Martin Seilmayer <M.Seilmayer at hzdr.de> wrote:
>> Hi all of you,
>>
>> I successfully created a hybrid cluster of several Windows and Linux
>> machines using snow and MPICH2. Basically I setup a SOCK - Cluster. To start
>> the Rscript processes on each machine MPICH2 comes in the game. Because it
> Are you saying that you're using MPICH2 to start your workers on the
> remote machines when creating a SOCK cluster? How are you doing that?
> Are you setting the rshcmd option in some way? Does MPICH2 include a
> remote execution command that is ssh-like?
>
> - Steve Weston
More information about the R-sig-hpc
mailing list