[R-sig-hpc] parallel code using snowfall

[Brian G. Peterson]

On 03/24/2010 05:31 AM, Carl Boettiger wrote:
> Dear R-sig-phylo,
>
> Is there a way to export all functions from a particular namespace in
> sfExport?
>
> I find I can export all functions I have declared using sfExportAll(), but
> functions I have imported from a library I have to export one by one:
>
> sfExport("fn1", namespace="packagename")
> sfExport("fn2", namespace="packagename")
> sfExport("fn3", namespace="packagename2")
>
> Also, is it computationally inefficient to export more functions than
> necessary for the computation?  Thanks again for the help,
>    

The 'foreach' package will automatically export any variables or 
functions that execute inside the foreach 'loop' construct.  Others that 
are not obviously needed (such as packages with hidden dependencies) may 
be explicitly exported.  foreach has a 'doSNOW' parallel backend.

It is not 'computationally inefficient' per se to export more than 
required, but it does take time to register all those objects, and to 
move them across the network.  So it is generally best to send only what 
you need to workers, and focus their activities on CPU-bound tasks, 
returning summary data that may be recombined and manuipulated by the 
master node.

Regards,

     - Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock