[R-sig-hpc] Can't start multi node SNOW cluster
Xiaobo Gu
guxiaobo1982 at gmail.com
Sun Feb 5 05:35:14 CET 2012
It seems there is a bug in snow or the samle code,
It seems snow treats every entry in the sep list as a seperate worker
> winOptions <-
+ list(host="192.168.72.1",
+ rscript="D:/Amber/Program/R/R-2.14.1/bin/x64/Rscript.exe",
+ snowlib="C:/Users/dell/Documents/R/win-library/2.14")
> cl <- makeCluster(c(rep(winOptions, 2)), type = "SOCK", manual=TRUE)
Manually start worker on 192.168.72.1 with
D:/Amber/Program/R/R-214~1.1/bin/Rscript.exe C:/Users/dell/Documents/R/win-library/2.14/snow/RSOCKnode.R MASTER=DELL-PC PORT=10187 OUT=D:/rworkout.txt SNOWLIB=C:/Users/dell/Documents/R/win-library/2.14
Manually start worker on D:/Amber/Program/R/R-2.14.1/bin/x64/Rscript.exe with
D:/Amber/Program/R/R-214~1.1/bin/Rscript.exe C:/Users/dell/Documents/R/win-library/2.14/snow/RSOCKnode.R MASTER=DELL-PC PORT=10187 OUT=D:/rworkout.txt SNOWLIB=C:/Users/dell/Documents/R/win-library/2.14
Manually start worker on C:/Users/dell/Documents/R/win-library/2.14 with
D:/Amber/Program/R/R-214~1.1/bin/Rscript.exe C:/Users/dell/Documents/R/win-library/2.14/snow/RSOCKnode.R MASTER=DELL-PC PORT=10187 OUT=D:/rworkout.txt SNOWLIB=C:/Users/dell/Documents/R/win-library/2.14
Manually start worker on 192.168.72.1 with
D:/Amber/Program/R/R-214~1.1/bin/Rscript.exe C:/Users/dell/Documents/R/win-library/2.14/snow/RSOCKnode.R MASTER=DELL-PC PORT=10187 OUT=D:/rworkout.txt SNOWLIB=C:/Users/dell/Documents/R/win-library/2.14
Manually start worker on D:/Amber/Program/R/R-2.14.1/bin/x64/Rscript.exe with
D:/Amber/Program/R/R-214~1.1/bin/Rscript.exe C:/Users/dell/Documents/R/win-library/2.14/snow/RSOCKnode.R MASTER=DELL-PC PORT=10187 OUT=D:/rworkout.txt SNOWLIB=C:/Users/dell/Documents/R/win-library/2.14
Manually start worker on C:/Users/dell/Documents/R/win-library/2.14 with
D:/Amber/Program/R/R-214~1.1/bin/Rscript.exe C:/Users/dell/Documents/R/win-library/2.14/snow/RSOCKnode.R MASTER=DELL-PC PORT=10187 OUT=D:/rworkout.txt SNOWLIB=C:/Users/dell/Documents/R/win-library/2.14
> clusterApply(cl, 1:2, get("+"), 3)
[[1]]
[1] 4
[[2]]
[1] 5
> stopCluster(cl)
Xiaobo Gu
From: Xiaobo Gu
Date: 2012-02-04 22:51
To: r-sig-hpc
Subject: Can't start multi node SNOW cluster
Hi,
First of all, my environment is :
64 bit Windows 7 Home basic with firewall off and the IP address 192.168.72.1 , R 2.14.1 64 bit, snow0.3.8 and doSNOW1.0.5
The first test senario works:
cl <- makeCluster(4, type = "SOCK")
clusterApply(cl, 1:2, get("+"), 3)
stopCluster(cl)
Before start the two node cluster I try to use the multi node syntex on the local machine,
winOptions <-
list(host="192.168.72.1",
rscript="D:/Amber/Program/R/R-2.14.1/bin/x64/Rscript.exe",
snowlib="C:/Users/dell/Documents/R/win-library/2.14")
cl <- makeCluster(c(rep(winOptions, 2)), type = "SOCK")
The makeCluster call just hung.
Then I do the same test on the single node 64bit CENTOS LINUX system, using the same versions of R, snow and doSNOW
Again, cl <- makeCluster(4, type = "SOCK") works,
lixOptions <-
list(host="hdp1",
rscript="/opt/r2141/lib64/R/bin/Rscript",
snowlib="/opt/r2141/lib64/R/library")
> cl <- makeCluster(c(rep(lixOptions,2)), type="SOCK")
The authenticity of host '192.168.72.7 (192.168.72.7)' can't be established.
RSA key fingerprint is 03:61:3b:4f:72:60:a7:a3:2a:8f:25:16:be:02:56:7c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.72.7' (RSA) to the list of known hosts.
ssh: /opt/r2141/lib64/R/bin/Rscript: Name or service not known
> cl <- makeCluster(c(rep(lixOptions,2)), type="SOCK")
ssh: /opt/r2141/lib64/R/bin/Rscript: Name or service not known
Can you help find what's wrong here.
There is another question about multi computer snow clusters,
becuase the master process must log onto remote machine to start worker process, where can I specify the user name and password, in modern Windows server environments uers must log on first before doing anything on it.
Regards,
Xiaobo Gu
More information about the R-sig-hpc
mailing list