Hi, Toby,

Thanks for the report. Off the top of my head I don't know why you're
seeing what you're seeing, but I will investigate. It does seem like a
problem with doParallel.

Cheers,
Rich Calaway
Revolution Analytics


On Mon, Jan 21, 2013 at 2:12 AM, Toby Hocking <toby.hocking@inria.fr> wrote:

> Hi all, I would appreciate some advice that could help me get my cluster
> to work with foreach. Essentially I do
>
> 1. makePSOCKcluster on 2 remote nodes, clusterCall => works
>
> 2. registerDoParallel, foreach => error in unserialize
>
> 3. clusterCall => does not work anymore, error in serialize
>
> 4. makePSOCKcluster only at localhost, clusterCall, foreach => works
>
> Any ideas why foreach is giving me this nasty error? I guess there is some
> state in the cl variable which is being corrupted? Is there any workaround?
>
> For future reference, are there any tricks for debugging problems like
> this?
>
> > require(parallel)
> Loading required package: parallel
> > cl <- makePSOCKcluster(c("sim18.sugi-lab","sim19.sugi-lab"),
> +                        user="hocking",
> +                        rscript="/home/hocking/lib64/R/bin/Rscript",
> +                        homogeneous=FALSE,
> +                        master="192.168.224.47")
> > hostname <- function(...)system("hostname",intern=TRUE)
> > clusterCall(cl,hostname) #works
> [[1]]
> [1] "sim18"
>
> [[2]]
> [1] "sim19"
>
> > clusterApply(cl,1:2,hostname) #works
> [[1]]
> [1] "sim18"
>
> [[2]]
> [1] "sim19"
>
> > showConnections()
>   description           class      mode  text     isopen   can read can
> write
> 3 "<-sim18.local:10187" "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> 4 "<-sim19.local:10187" "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> > require(doParallel)
> Loading required package: doParallel
> Loading required package: foreach
> foreach: simple, scalable parallel programming from Revolution Analytics
> Use Revolution R for scalability, fault tolerance and more.
> http://www.revolutionanalytics.com
> Loading required package: iterators
> > registerDoParallel(cl)
> > foreach(i=1:2)%dopar%{
> +   hostname()
> + }
> Error in unserialize(node$con) : error reading from connection
> > clusterCall(cl,hostname) #doesn't work
> Error in serialize(data, node$con) : ignoring SIGPIPE signal
> > clusterCall(cl,hostname) #doesn't work
> Error in serialize(data, node$con) : error writing to connection
> > clusterCall(cl,hostname) #doesn't work
> Error in serialize(data, node$con) : error writing to connection
> > showConnections()
>   description           class      mode  text     isopen   can read can
> write
> 3 "<-sim18.local:10187" "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> 4 "<-sim19.local:10187" "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> > cl <- makePSOCKcluster(rep("localhost",2))
> > clusterCall(cl,hostname) #works
> [[1]]
> [1] "toby-CF-SX2MEKBP"
>
> [[2]]
> [1] "toby-CF-SX2MEKBP"
>
> > clusterApply(cl,1:2,hostname) #works
> [[1]]
> [1] "toby-CF-SX2MEKBP"
>
> [[2]]
> [1] "toby-CF-SX2MEKBP"
>
> > registerDoParallel(cl)
> > foreach(i=1:2)%dopar%{
> +   hostname()
> + } #works
> [[1]]
> [1] "toby-CF-SX2MEKBP"
>
> [[2]]
> [1] "toby-CF-SX2MEKBP"
>
> > clusterCall(cl,hostname) #works
> [[1]]
> [1] "toby-CF-SX2MEKBP"
>
> [[2]]
> [1] "toby-CF-SX2MEKBP"
>
> > showConnections()
>   description           class      mode  text     isopen   can read can
> write
> 3 "<-sim18.local:10187" "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> 4 "<-sim19.local:10187" "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> 5 "<-localhost:10187"   "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> 6 "<-localhost:10187"   "sockconn" "a+b" "binary" "opened" "yes"
> "yes"
> > sessionInfo()
> R version 2.15.2 (2012-10-26)
> Platform: x86_64-unknown-linux-gnu (64-bit)
>
> locale:
>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>  [7] LC_PAPER=C                 LC_NAME=C
>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] parallel  graphics  utils     datasets  stats     grDevices methods
> [8] base
>
> other attached packages:
> [1] doParallel_1.0.1   iterators_1.0.6    foreach_1.4.0
> ggplot2_0.9.3
> [5] RColorBrewer_1.0-5 lattice_0.20-10
>
> loaded via a namespace (and not attached):
>  [1] codetools_0.2-8  colorspace_1.2-0 compiler_2.15.2  dichromat_1.2-4
>  [5] digest_0.6.0     grid_2.15.2      gtable_0.1.2     labeling_0.1
>  [9] MASS_7.3-22      munsell_0.4      plyr_1.8         proto_0.3-10
> [13] reshape2_1.2.2   scales_0.2.3     stringr_0.6.2
> >
>



-- 
Rich Calaway
Documentation Manager
Revolution Analytics, Inc.
1505 Westlake Ave North Suite 300
Seattle, WA 98109
richcalaway@revolutionanalytics.com
206-456-6086 (Direct Phone)
855-GET-REVO x6086 (Toll-free)

	[[alternative HTML version deleted]]

