I hadn't seen that, thanks. It looks like a nice mechanism for passing
messages and sharing data between multiple clients. It would be interesting
if someone created an R environment based on a dynamic object tables-based
backend that shared data via redis. I don't see anything about managing of
resource pools though.

Michael

On Wed, Nov 14, 2012 at 8:20 AM, Cook, Malcolm <MEC@stowers.org> wrote:

> Michael,
>
> Have you seen  http://cran.r-project.org/web/packages/doRedis/index.html??
>
> If you take a look and come across a description of
> internals/architecture, please share....
>
> Cheers,
>
> ~Malcolm
>
>
> > -----Original Message-----
> > From: bioconductor-bounces@r-project.org [mailto:
> bioconductor-bounces@r-project.org] On Behalf Of Michael Lawrence
> > Sent: Wednesday, November 14, 2012 8:41 AM
> > To: Bioconductor List
> > Subject: [BioC] pooling for parallel hierarchical operations
> >
> > We often execute nested operations in parallel. For example, first by
> > sample, then by chromosome. Fixed allocation of resources to each level
> > will often result in waste. For example, if one sample finishes quickly,
> > its CPUs are not available to help the other samples along. Perhaps the
> > most expedient solution is to expand.grid() the hierarchy and create one
> > job for every combination, i.e., flatten the hierarchy. A more ideal
> > solution might be a pool of resources (cores) that are allocated more
> > fluidly. Is there any sort of pooling system for R? I know that the
> > parallel package supports the declaration of resources in cluster
> objects,
> > but there is no central manager. This is a general R question, but it's
> > worth discussing in the context of how we can make better use of
> > parallelism in the low-level infrastructure, which would cause these
> > hierarchies to arise. It's also relevant to the discussion of specifying
> > parallelization modes or strategies. Pools themselves could be
> hierarchical
> > and heterogeneous (hosts, cores). Declaring available resources is fairly
> > straight-forward. Deciding how to use them is context dependent and
> > requires user control.
> >
> > Michael
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioconductor mailing list
> > Bioconductor@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/bioconductor
> > Search the archives:
> http://news.gmane.org/gmane.science.biology.informatics.conductor
>

	[[alternative HTML version deleted]]

