[R-sig-hpc] SNOW Issues: Error in writing to connection (on localhost)

Cedrick Johnson cedrick at cedrickjohnson.com
Mon Mar 14 05:10:22 CET 2011

Hi All-
I am having trouble on a new machine with a fresh install of R and the
requisite SNOW packages (snow, foreach, doSNOW). When I attempt to
start 2 instances on localhost and use the %dopar% utility, I get the
following message:

Error in serialize(data, node$con) : error writing to connection

Here's that systems' details:
OS: Linux Mint 10 (Ubuntu 10.10)
SessionInfo (relevant packages):

other attached packages:
 [1] doSNOW_1.0.3                 foreach_1.3.0
 [4] iterators_1.0.3              fogbank_2.1.0
 [7] fExoticOptions_2110.77       fOptions_2110.78             snow_0.3-3

When I look at the created cluster object:
> cl.tmp
                    description                           class
"<-localhost.localdomain:10187"                      "sockconn"
                           mode                            text
                          "a+b"                        "binary"
                         opened                        can read
                       "opened"                           "yes"
                      can write


On another system, the actual hostname of the system is shown (and I'm
able to run the code successfully):

> cl.tmp2
                    description                           class
"<-fogb-chi-cj1:10187"                      "sockconn"
                           mode                            text
                          "a+b"                        "binary"
                         opened                        can read
                       "opened"                           "yes"
                      can write

Here's what I'm trying to run. Unfortunately, the data is too large to attach:

xj = colnames(HistoricalYields)


cl.tmp = makeCluster(rep("localhost",2), type="SOCK")

parallel.arima <- function(data) {
	fit = auto.arima(ts(HistoricalYields[,data]), approximation=TRUE,
allowdrift=TRUE, stepwise=TRUE)
system.time(res <- foreach(dat=xj) %dopar% parallel.arima(dat))

** Error (as above)

One more note, this appears to work for the single case. I'm not quite
sure what the ramifications would be if I tried running this with
distributed nodes (I'm going to test that out in a few minutes to

clusterExport(cl.tmp, c("HistoricalYields",xj))
system.time(res <- clusterApply(cl.tmp, fun=parallel.arima, xj))

Thanks for any advice/help.. Should I be using clusterExport and
bypassing %dopar% altogether?


More information about the R-sig-hpc mailing list