[R-SIG-Mac] multicore package: collecting results
bbolker at gmail.com
Wed Jun 29 20:47:55 CEST 2011
-----BEGIN PGP SIGNED MESSAGE-----
On 06/29/2011 02:34 PM, Vincent Aubanel wrote:
> Hi all,
> I'm using mclapply() of the multicore package for processing chunks
> of data in parallel --and it works great.
> But when I want to collect all processed elements of the returned
> list into one big data frame it takes ages.
> The elements are all data frames having identical column names, and
> I'm using a simple rbind() inside a loop to do that. But I guess it
> makes some expensive checking computations at each iteration as it
> gets slower and slower as it goes. Writing out to disk individual
> files, concatenating with the system and reading back from disk the
> resulting file is actually faster...
Why do you have to write to disk? Can you collect the results as a
list L and then do.call(rbind,L) in one go?
> Is there a magic argument to rbind() that I'm missing, or is there
> any other solution to collect the results of parallel processing
> Thanks, Vincent
> _______________________________________________ R-SIG-Mac mailing
> list R-SIG-Mac at r-project.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
-----END PGP SIGNATURE-----
More information about the R-SIG-Mac