[R-sig-hpc] snow package and architecture

[Stuart Andrews]

Hi,

I have been mostly successful using the snow package to take advantage  
of multiples cores on a MacPro running Leopard OS X 10.5.8, as follows.

 > cl <- makeSOCKcluster(rep("mac133990", 3))

One difficulty, however, was that the slave nodes are starting up in  
32-bit mode despite the fact that the master is running in 64-bit  
mode.  Note that when I start the cluster manually  
(setDefaultClusterOptions(manual=TRUE)), I get a 64-bit cluster!    
However, I cannot get this to work otherwise.

I  suspect the environment variable R_ARCH, from my .bash_profile, is  
not being passed to the child processes.  These are spawned from the  
snow function newSOCKnode() via a system command, prefixed with "ssh - 
l USER MACHINE ..."

How can I get the snow cluster to recognize my R_ARCH without starting  
it up manually every time?

I've tried ...

(1) adding R_ARCH to my .Renviron file
(2) changing the defaultClusterOptions variable:  rshcmd="R_ARCH=/ 
x86_64; ssh"
(3) changing the defaultClusterOptions variable:  rshcmd="R_ARCH=/ 
x86_64; export R_ARCH; ssh"
(4) adding R_ARCH to my ~/.ssh/environment

But each time, I end up with a 32-bit cluster!

Thanks
- Stu



mac133990 ~/Data/methyl-091005/R -> Rmac

R version 2.11.0 Under development (unstable) (2010-01-14 r50976)
Copyright (C) 2010 The R Foundation for Statistical Computing
ISBN 3-900051-07-0

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

   Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

PWD->[ /Volumes/ScratchDisk/Data/methyl-091005/R ]


 > .Machine$sizeof.pointer
[1] 8

 > library(snow)

 > setDefaultClusterOptions(manual=F)

 > cl=makeSOCKcluster(rep("mac133990",3))
Warning: No xauth data; using fake authentication data for X11  
forwarding.
PWD->[ /Users/stu ]
Warning: No xauth data; using fake authentication data for X11  
forwarding.
PWD->[ /Users/stu ]
Warning: No xauth data; using fake authentication data for X11  
forwarding.
PWD->[ /Users/stu ]


 > clusterEvalQ(cl, .Machine$sizeof.pointer)
[[1]]
[1] 4

[[2]]
[1] 4

[[3]]
[1] 4

 > stopCluster(cl)



 > setDefaultClusterOptions(manual=T)

 > cl=makeSOCKcluster(rep("mac133990",2))
Manually start worker on mac133990 with
      /Library/Frameworks/R.framework/Resources/bin/Rscript /Users/stu/ 
Research/Tools/R_library_version.2.11.0_arch.x86_64/snow/RSOCKnode.R  
MASTER=mac133990.med.cornell.edu PORT=10187 OUT=/dev/null SNOWLIB=/ 
Users/stu/Research/Tools/R_library_version.2.11.0_arch.x86_64
Manually start worker on mac133990 with
      /Library/Frameworks/R.framework/Resources/bin/Rscript /Users/stu/ 
Research/Tools/R_library_version.2.11.0_arch.x86_64/snow/RSOCKnode.R  
MASTER=mac133990.med.cornell.edu PORT=10187 OUT=/dev/null SNOWLIB=/ 
Users/stu/Research/Tools/R_library_version.2.11.0_arch.x86_64


 > clusterEvalQ(cl, .Machine$sizeof.pointer)
[[1]]
[1] 8

[[2]]
[1] 8

 > sessionInfo()
R version 2.11.0 Under development (unstable) (2010-01-14 r50976)
x86_64-apple-darwin9.8.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] snow_0.3-3  gdata_2.6.1

loaded via a namespace (and not attached):
[1] gtools_2.6.1