[Bioc-devel] BiocParallel

[Hahne, Florian]

I'll take a look at this. Until now I was mainly playing around with the
backend part and I don't have a strong opinion on any parallel framework
yet. The parLapply solution seemed the one closest to R's functional
paradigm, but it should be fairy trivial to stick this into something
else, including %dopar%.
Florian
-- 






On 11/16/12 6:41 PM, "Ryan C. Thompson" <rct at thompsonclan.org> wrote:

>You could write a %dopar% backend for the foreach package, which would
>allow any code using foreach (or plyr which uses foreach) to parallelize
>using your code.
>
>On a related note, it might be nice to add Bioconductor-compatible
>versions of foreach and the plyr functions to BiocParallel if they're
>not already compatible.
>
>On 11/16/2012 12:18 AM, Hahne, Florian wrote:
>> I've hacked up some code that uses BatchJobs but makes it look like a
>> normal parLapply operation. Currently the main R process is checking the
>> state of the queue in regular intervals and fetches results once a job
>>has
>> finished. Seems to work quite nicely, although there certainly are more
>> elaborate ways to deal with the synchronous/asynchronous issue. Is that
>> something that could be interesting for the broader audience? I could
>>add
>> the code to BiocParallel for folks to try it out.
>> The whole thing may be a dumb idea, but I find it kind of useful to be
>> able to start parallel jobs directly from R on our huge SGE cluster,
>>have
>> the calling script wait for all jobs to finish and then continue with
>>some
>> downstream computations, rather than having to manually check the job
>> status and start another script once the results are there.
>> Florian
>