[R-sig-hpc] new doMPI package available on CRAN
Stephen Weston
stephen.b.weston at gmail.com
Wed Dec 9 16:49:44 CET 2009
I'd like to announce the availability of the new "doMPI" package, a
parallel backend for the "foreach" package, which acts as an adaptor to
the "Rmpi" package. The package has been uploaded to CRAN and is now
available.
I wrote this package after leaving REvolution Computing because I liked
the foreach package, wanted to learn more about the Rmpi package, and
thought it might be of value to the R community by combining the speed
of Rmpi with the ease of use of foreach. Perhaps that's an odd way to
spend my time when my wife probably thinks I should be looking for a job.
I guess I'm obsessive. And I thought it would only take a week or two.
I'm not really sure if I succeeded in making a package that is substantially
better than the doSNOW package that I wrote previously, at least on smaller
clusters where broadcasting isn't very helpful. I ended up adding tuning
arguments that allow broadcasting to be turned off, force the use of piggy-
backing shared data with the first task, etc. At any rate, I learned a lot,
had fun, and gained an even deeper appreciation of the excellent snow
package during my testing and benchmarking.
I would like to thank Dirk Eddelbuettel and Bryan Lewis for their help
on the doMPI package. Both found problems in the package, and gave me
valuable feedback. I was also greatly helped by Dirk's excellent "HPC
with R" tutorial. It has information on using Rmpi that I didn't find
anywhere else. Bryan researched various MPI topics for me, and gave me
access to an excellent little cluster for testing purposes.
Also note that the package includes a vignette which has information
on running doMPI scripts using the orterun command. I'm tempted to
write a similar article for snow, since there doesn't seem to be much
information available on that subject outside of Dirk's tutorial.
- Steve Weston
More information about the R-sig-hpc
mailing list