[R-sig-hpc] new doMPI package available on CRAN

Stephen Weston stephen.b.weston at gmail.com
Wed Dec 9 16:49:44 CET 2009


I'd like to announce the availability of the new "doMPI" package, a
parallel backend for the "foreach" package, which acts as an adaptor to
the "Rmpi" package.  The package has been uploaded to CRAN and is now
available.

I wrote this package after leaving REvolution Computing because I liked
the foreach package, wanted to learn more about the Rmpi package, and
thought it might be of value to the R community by combining the speed
of Rmpi with the ease of use of foreach.  Perhaps that's an odd way to
spend my time when my wife probably thinks I should be looking for a job.
I guess I'm obsessive.  And I thought it would only take a week or two.

I'm not really sure if I succeeded in making a package that is substantially
better than the doSNOW package that I wrote previously, at least on smaller
clusters where broadcasting isn't very helpful.  I ended up adding tuning
arguments that allow broadcasting to be turned off, force the use of piggy-
backing shared data with the first task, etc.  At any rate, I learned a lot,
had fun, and gained an even deeper appreciation of the excellent snow
package during my testing and benchmarking.

I would like to thank Dirk Eddelbuettel and Bryan Lewis for their help
on the doMPI package.  Both found problems in the package, and gave me
valuable feedback.  I was also greatly helped by Dirk's excellent "HPC
with R" tutorial.  It has information on using Rmpi that I didn't find
anywhere else.  Bryan researched various MPI topics for me, and gave me
access to an excellent little cluster for testing purposes.

Also note that the package includes a vignette which has information
on running doMPI scripts using the orterun command.  I'm tempted to
write a similar article for snow, since there doesn't seem to be much
information available on that subject outside of Dirk's tutorial.

- Steve Weston



More information about the R-sig-hpc mailing list