rdiaz02 at gmail.com
Sat Nov 17 11:39:55 CET 2012
On Sat, 17 Nov 2012 00:45:46 -0500,Steve Lianoglou <mailinglist.honeypot at gmail.com> wrote:
> On Fri, Nov 16, 2012 at 11:42 PM, Ryan C. Thompson <rct at thompsonclan.org> wrote:
> > The difference is that in the parallel package, you use mclapply for
> > multicore and parLapply for multi-machine parallelism. If you want to switch
> > from one to the other, you have to change all your code that uses either
> > function to the other one. If you use llply(..., .parallel=TRUE), then all
> > you have to do is register a different backend, which is one line of code to
> > load the new backend and a second one to register it, and the rest of your
> > code stays the same.
In addition to Steve's comment, is it really a good thing that "all code
stays the same."? I mean, multiple machines vs. multiple cores are,
often, _very_ different things: for instance, shared vs. distributed
memory, communication overhead differences, whether or not you can assume
packages and objects to be automagically present in the slaves/child
process, etc. So, given they are different situations, I think it
sometimes makes sense to want to write different code for each situation
(I often do); not to mention Steve's hybrid cases ;-).
Since BiocParallel seems to be a major undertaking, maybe it would be
appropriate to provide a flexible approach, instead of hard wiring the
> This is the same reason I've also floated use of foreach in the past.
> But then you have the situation of multi-machines w/ multiple cores --
> is this (2) or (3) here?
> How do you explicitly write code for that w/ foreach mojo? I guess the
> answer to that is that you let your "grid engine" (or whatever your
> SGE equivalent is that you might be running) handle that for you.
> Steve Lianoglou
> Graduate Student: Computational Systems Biology
> | Memorial Sloan-Kettering Cancer Center
> | Weill Medical College of Cornell University
> Contact Info: http://cbio.mskcc.org/~lianos/contact
> Bioc-devel at r-project.org mailing list
Department of Biochemistry, Lab B-25
Facultad de Medicina
Universidad Autónoma de Madrid
Arzobispo Morcillo, 4
Email: rdiaz02 at gmail.com
ramon.diaz at iib.uam.es
More information about the Bioc-devel