[Bioc-devel] reporting progress while running in parallel

Martin Morgan mtmorgan at fhcrc.org
Sat Aug 28 05:04:52 CEST 2010


On 08/27/2010 07:27 AM, Robert Castelo wrote:
> dear developers,
> 
> i'm adding parallel support to my package by copying & pasting the way
> in which this is done by other packages like ShortRead. i'm currently
> using 'snow' and since i'm providing this support for
> computing-intensive calculations i'd like that while these calculations
> are running in parallel, some sort of progress is reported.
> 
> i'd like to ask whether anybody here has added parallel support to
> his/her package including some sort of progress reporting functionality.
> 
> my first guess is that probably the loop of slave R processes created by
> 'snow' should be replaced by something else, but i hope somebody has a
> more clear idea about what could be done.

Hi Robert --

I don't have a solution, but if I were starting on this with snow I'd
look at clusterApplyLB and dynamicClusterApply -- break the overall set
of tasks into a number of jobs > the number of nodes, and then in a loop
like that in dynamicClusterApply report to the user as each job comes
back from recvOneResult.

In Rmpi a fun project might be to figure out how to use a separate comm
to send errors.

Martin

> 
> 
> thanks!
> robert.
> 
> _______________________________________________
> Bioc-devel at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel


-- 
Martin Morgan
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793



More information about the Bioc-devel mailing list