[R-sig-hpc] SNOW Issues: Error in writing to connection (on localhost)
Cedrick Johnson
cedrick at cedrickjohnson.com
Mon Mar 14 05:10:22 CET 2011
Hi All-
I am having trouble on a new machine with a fresh install of R and the
requisite SNOW packages (snow, foreach, doSNOW). When I attempt to
start 2 instances on localhost and use the %dopar% utility, I get the
following message:
Error in serialize(data, node$con) : error writing to connection
Here's that systems' details:
OS: Linux Mint 10 (Ubuntu 10.10)
SessionInfo (relevant packages):
other attached packages:
[1] doSNOW_1.0.3 foreach_1.3.0
codetools_0.2-8
[4] iterators_1.0.3 fogbank_2.1.0
XML_3.2-0
[7] fExoticOptions_2110.77 fOptions_2110.78 snow_0.3-3
When I look at the created cluster object:
> cl.tmp
[[1]]
$con
description class
"<-localhost.localdomain:10187" "sockconn"
mode text
"a+b" "binary"
opened can read
"opened" "yes"
can write
"yes"
***************
On another system, the actual hostname of the system is shown (and I'm
able to run the code successfully):
> cl.tmp2
[[1]]
$con
description class
"<-fogb-chi-cj1:10187" "sockconn"
mode text
"a+b" "binary"
opened can read
"opened" "yes"
can write
"yes"
Here's what I'm trying to run. Unfortunately, the data is too large to attach:
xj = colnames(HistoricalYields)
require(snow)
require(doSNOW)
require(foreach)
cl.tmp = makeCluster(rep("localhost",2), type="SOCK")
registerDoSNOW(cl.tmp)
parallel.arima <- function(data) {
library(forecast)
fit = auto.arima(ts(HistoricalYields[,data]), approximation=TRUE,
allowdrift=TRUE, stepwise=TRUE)
}
system.time(res <- foreach(dat=xj) %dopar% parallel.arima(dat))
** Error (as above)
One more note, this appears to work for the single case. I'm not quite
sure what the ramifications would be if I tried running this with
distributed nodes (I'm going to test that out in a few minutes to
see):
clusterExport(cl.tmp, c("HistoricalYields",xj))
system.time(res <- clusterApply(cl.tmp, fun=parallel.arima, xj))
Thanks for any advice/help.. Should I be using clusterExport and
bypassing %dopar% altogether?
Thanks,
Cedrick
More information about the R-sig-hpc
mailing list