[R] multi-machine parallel setup?

[Uwe Ligges]

On 21.04.2012 17:20, ivo welch wrote:
> hi uwe---my box is different:
>
> cl<- makeCluster(c("localhost", "welch", "calc"), "SOCK")
> Error in loadNamespace(name) : there is no package called 'snow'
>
> this seems to be part of snow, not parallel.  ok, install snow first,
> then rerun.


No, you just need snow to be installed, using the functions from 
parallel will load the snow Namespace, if installed.




> library(snow)
> Attaching package: 'snow'
>
> The following object(s) are masked from 'package:parallel':
>
>      clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
> clusterExport, clusterMap, clusterSplit, makeCluster, parApply,
> parCapply, parLapply,
>      parRapply, parSapply, splitIndices, stopCluster
>
> bash: /Library/Frameworks/R.framework/Resources/bin/Rscript: No such
> file or directory
>
> but if I use two linux machines, it works.
>
> now, how do I use parallel's mclapply with it?


as I said.

As you seem to be under some heterogeneous setup (unstated before, as 
far as I remember), you may need to specify paths to the loaction of R 
etc. See the documentation for the parallel package.

Uwe



> best,
>
> /iaw
>
> ----
> Ivo Welch (ivo.welch at gmail.com)
> J. Fred Weston Professor of Finance
> Anderson School at UCLA, C519
> http://www.ivo-welch.info/
> Editor, Critical Finance Review, http://www.critical-finance-review.org/
>
>
>
> 2012/4/21 Uwe Ligges<ligges at statistik.tu-dortmund.de>:
>>
>>
>> On 21.04.2012 06:34, ivo welch wrote:
>>>
>>> the vignette to the library(parallel) mentions snow repeatedly (esp
>>> differences in its implementation in parallel from the original).
>>> unfortunately, it doesn't give an example or tutorial for
>>> multi-machine use with sockets.
>>>
>>> could someone please point me to a simple working example, where a
>>> master just has both itself and a 4-core slave on a second machine
>>> (IP), and wants to execute an mclapply on a list on both machines,
>>> like
>>>
>>>     master.ip<- "localhost"
>>>     slave.ip<- "Rfriend at 192.168.2.10"  ## has key ssh access
>>>     mclapply( 1:100, function(x) { cat(x); Sys.sleep(1); x } )  # on both,
>>> please
>>>
>>> iaw
>>
>>
>>
>>
>> With appropriate ssh keys that do not need passwords and with the same
>> username on all machines, it should work out of the box. At least, it does
>> for me:
>>
>> Say you want two, one on your own machine, the other one on your friends:
>>
>>   cl<- makeCluster(c("localhost", "192.168.2.10"), "SOCK")
>>
>> BEst,
>> Uwe Ligges
>>
>>
>>>
>>> On Wed, Apr 18, 2012 at 1:01 PM, ivo welch<ivo.welch at gmail.com>    wrote:
>>>>
>>>> Dear R experts:
>>>>
>>>> could someone please point me to a page that explains how to set up
>>>> more than 1 machine for library parallel (which is quickly becoming my
>>>> favorite!)
>>>
>>> ...
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.