I've done quite a bit of this nested parallelism with BatchJobs/SGE over
multicore.

I daresay I haven't looked at BiocParallel as the solutions seemed pretty
obvious
with standard calls.  Should BiocParallel have a vignette that indicates
added value?

I would like us to have a worked demonstration in an AMI for a nested
parallel solution.
The "Cloud-enabled cis-eQTL..." workflow document gives some gory details
for snow/multicore
in EC2 using starcluster to manage the AMIs, and identifies an AMI where
the process can be demonstrated.  I would like to control the process via
BatchJobs but I haven't been able to get SGE running in EC2 ...


On Fri, Feb 14, 2014 at 12:28 PM, Michael Lawrence <
lawrence.michael@gene.com> wrote:

> On Fri, Feb 14, 2014 at 9:20 AM, Laurent Gatto <lg390@cam.ac.uk> wrote:
>
> >
> > Dear all,
> >
> > Looking back in my (sparse) notes from the last European Bioc meeting, I
> > saw two points that were mentioned during the BatchJobs and BiocParallel
> > talk that are of particular interest to me.
> >
> > - BatchJobs: is there support, or are there any plans to support the
> >   HTCondor infrastructure [1], and will this back-end be supported in
> >   BiocParallel?
> >
> >
> If BatchJobs supports it, then so will BiocParallel, thanks to the
> abstraction.
>
>
> > - BiocParallel: is there support, or are there any plans to support
> >   nested parallelisation. The use case I have in mind is to have a first
> >   level of parallelisation to distribute n sets jobs on n different
> >   machines using snow for example, and then each set of jobs would be
> >   distributed on their respective machine via multicore?
> >
> >
> Yes, this is done by passing a list of BiocParallelParam objects to
> bplapply. Not sure if it's ever been tested though.
>
>
> > Any comments on the above solutions or their feasibility would be very
> > welcome.
> >
> > Thank you very much in advance.
> >
> > Best wishes,
> >
> > Laurent
> >
> > [1]  http://research.cs.wisc.edu/htcondor/
> >
> >
> > --
> > Laurent Gatto
> > http://cpu.sysbiol.cam.ac.uk
> > http://proteomics.bio.cam.ac.uk
> >
> > _______________________________________________
> > Bioc-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]

