[R-sig-hpc] problem with opening more than one SOCK cluster with package snow

[Jannis]

Hi Steve,

thanks a lot for your reply. There may be an elegant and fail proof 
method for port determination but I learned during my work that 
sometimes the quick and dirty way has a much higher output/programming 
energy ratio ;-). So I would start with assigning random ports each time 
a cluster is started and restart the odd jobs with conflicting ports. I 
am no expert in ports and related things, but are there possible 
problems with other programs or even security that might occour if I use 
random port numbers between 1023 and 65535?


Thanks a lot
Jannis

On 20.12.2012 16:00, Stephen Weston wrote:
> Hi Jannis,
>
> I think you are right as to the cause of the error.  I believe that
> the problem occurs when two master processes start running
> on the same machine at about the same time because they
> will both try to bind to the same port by default.  Interestingly,
> the master process only binds to that port temporarily, which
> is why the problem is intermittent.  The more workers that you
> start, the longer it takes to create the cluster, and the more
> likely the error is to occur.
>
> I suggest that you use the "port" option when calling makeCluster,
> as in:
>
>    cl <- makeCluster(8, type="SOCK", port=10188)
>
> where you specify a different value for every master on the
> same machine.  I'm not aware of a fool-proof technique for
> doing that, but you can probably come up with some method
> that works pretty well for your situation.
>
> Of course, if you're not using Windows, you could try using
> the "parallel" package, and create a "FORK" cluster.  That
> doesn't use sockets at all, so multiple clusters on the same
> machine isn't a problem.
>
> - Steve
>
>
> On Thu, Dec 20, 2012 at 7:11 AM, Jannis <bt_jannis at yahoo.de> wrote:
>> Der HPC R users,
>>
>>
>> sorry for cross posting (to r-help) but I just realized after posting that
>> the hpc list may be more suitable for my request. I will report solutions to
>> both lists.
>>
>>
>> Cheers
>> Jannis
>>
>>
>> Dear r-help list,
>>
>> i have some problems using the snow package to create a SOCK cluster. The
>> errors just occour irregularly but it seems to me that they occour when I
>> try to create more than one cluster on the same machine via different R
>> instances started via submitting LSF jobs to a cluster. Does anyone have an
>> idea how to solve this or where to start digging for solutions?
>>
>> The error messages are:
>>
>> library(snow)
>> cl <- makeCluster(8, type = "SOCK")
>>
>>
>> Error in socketConnection(port = port, server = TRUE, blocking = TRUE,  :
>>    cannot open the connection
>> Calls: run.call ... makeCluster -> makeSOCKcluster -> newSOCKnode ->
>> socketConnection
>> In addition: Warning message:
>> In socketConnection(port = port, server = TRUE, blocking = TRUE,  :
>>    port 10187 cannot be opened
>>
>>
>> R version 2.15.1 (2012-06-22)
>> Platform: x86_64-unknown-linux-gnu (64-bit)
>>
>> locale:
>>   [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>>   [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>>   [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>>   [7] LC_PAPER=C                 LC_NAME=C
>>   [9] LC_ADDRESS=C               LC_TELEPHONE=C
>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>
>> attached base packages:
>> [1] stats     graphics  grDevices datasets  utils     methods   base
>>
>> other attached packages:
>> [1] snow_0.3-10
>>
>>
>> Thanks a lot
>> Jannis
>>
>> _______________________________________________
>> R-sig-hpc mailing list
>> R-sig-hpc at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc