[R] Rsge: recursive parallelization
Peter Danenberg
pcd at roxygen.org
Mon Apr 12 18:11:00 CEST 2010
Quoth Martin Morgan on Boomtime, the 29th of Discord:
> It seems like it would be hard to think about the tasks that are
> being executed, how many processes there are, how load balancing
> works, etc.
A flat representation of our nested data is, alas, too large to
contain in memory; and dynamically re-establishing context for an
arbitrary subset of it is expensive.
Parallel recursion isn't unheard of [1]; and I was hoping SGE would
worry about the load balancing, etc. for me.
Barring that, though, we might have to get creatively iterative.
Footnotes:
[1] http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=44263
More information about the R-help
mailing list