[Bioc-devel] BiocParallel

[Ryan C. Thompson]

In reply to:

On 11/16/2012 09:45 PM, Steve Lianoglou wrote:
> But then you have the situation of multi-machines w/ multiple cores -- 
> is this (2) or (3) here? How do you explicitly write code for that w/ 
> foreach mojo? I guess the answer to that is that you let your "grid 
> engine" (or whatever your SGE equivalent is that you might be running) 
> handle that for you. 

You just use makeCluster to spawn multiple workers per host.

library(parallel)
host.cores <- c(host1=4, host2=8, host3=2)
cl <- makeCluster(rep(names(host.cores), host.cores))
registerDoParallel(cl=cl)

Honestly, when I think about multi-machine parallelism, I generally 
assume that multiple cores per machine is implied. In any case, the 
point is that foreach supports any parallelization situation (multicore, 
multi-machine, both) for which a parallel backend exists. So of all the 
parallelism stuff that exists in R today, I think it's the closest thing 
we have to a general parallelization framework.