[R-sig-hpc] problem with opening more than one SOCK cluster with package snow

[Stephen Weston]

Hi Jannis,

I think you are right as to the cause of the error.  I believe that
the problem occurs when two master processes start running
on the same machine at about the same time because they
will both try to bind to the same port by default.  Interestingly,
the master process only binds to that port temporarily, which
is why the problem is intermittent.  The more workers that you
start, the longer it takes to create the cluster, and the more
likely the error is to occur.

I suggest that you use the "port" option when calling makeCluster,
as in:

  cl <- makeCluster(8, type="SOCK", port=10188)

where you specify a different value for every master on the
same machine.  I'm not aware of a fool-proof technique for
doing that, but you can probably come up with some method
that works pretty well for your situation.

Of course, if you're not using Windows, you could try using
the "parallel" package, and create a "FORK" cluster.  That
doesn't use sockets at all, so multiple clusters on the same
machine isn't a problem.

- Steve


On Thu, Dec 20, 2012 at 7:11 AM, Jannis <bt_jannis at yahoo.de> wrote:
> Der HPC R users,
>
>
> sorry for cross posting (to r-help) but I just realized after posting that
> the hpc list may be more suitable for my request. I will report solutions to
> both lists.
>
>
> Cheers
> Jannis
>
>
> Dear r-help list,
>
> i have some problems using the snow package to create a SOCK cluster. The
> errors just occour irregularly but it seems to me that they occour when I
> try to create more than one cluster on the same machine via different R
> instances started via submitting LSF jobs to a cluster. Does anyone have an
> idea how to solve this or where to start digging for solutions?
>
> The error messages are:
>
> library(snow)
> cl <- makeCluster(8, type = "SOCK")
>
>
> Error in socketConnection(port = port, server = TRUE, blocking = TRUE,  :
>   cannot open the connection
> Calls: run.call ... makeCluster -> makeSOCKcluster -> newSOCKnode ->
> socketConnection
> In addition: Warning message:
> In socketConnection(port = port, server = TRUE, blocking = TRUE,  :
>   port 10187 cannot be opened
>
>
> R version 2.15.1 (2012-06-22)
> Platform: x86_64-unknown-linux-gnu (64-bit)
>
> locale:
>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>  [7] LC_PAPER=C                 LC_NAME=C
>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices datasets  utils     methods   base
>
> other attached packages:
> [1] snow_0.3-10
>
>
> Thanks a lot
> Jannis
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc