[R-SIG-Mac] parallel processing with jags through R? - summary replies

[Alan Kelly]

Dear list, I received several replies to the above query from the r-sig-mac list (and these have been cc-ed to the list already) and subsequently several very specific replies from the BUGS list (details at http://www.mrc-bsu.cam.ac.uk/bugs/overview/list.shtml). As I've also received several requests to post information to this list, a few relevant extracts appear below my sig.

Following Bill Northcott's advice I've installed an optimised version of jags for 64 bit R on Mac (source mentioned below) and tried it out.  Very initial runs with subsets of my data suggest a time saving of between 10%-15%.  
I've also had a chance to try several examples of Peter Solymos' dclone package using the jags.parfit() function and these work perfectly.
Next steps: use the optimised jags from Bill Northcott in conjunction with Josh Nowak's suggestion re. snow and Simon Urbanek's re. multicore.  I also have the possibility of using runjags with Mac's xgrid cluster controller thanks to Guillaume Chapron.

My appreciation to all those who replied and I hope the ideas mentioned below are of use to other members of the list.
Alan Kelly
Trinity College Dublin

Extracts of some replies:

News of an optimised version of jags for 64 bit R on Mac OS 10.6  from Bill Northcott 
Are you using the optimised version which I built for 64 bit on 10.6.  It is much faster than the other package.
You could probably get even more speed if you build it yourself and use CPU flags for your computer.

(I've found this version at http://sourceforge.net/projects/mcmc-jags/files/JAGS/2.x/Mac%20OS%20X/)

+++++++++++++++++++++++++++++++++

from Elizabeth Tighe
There was a discussion on this issue a bit back on the jags forum:
http://sourceforge.net/projects/mcmc-jags/forums/forum/610037/topic/3887787
I've been using a routine (on linux ubuntu) that Mike Rutter shared 
using rmpi and rsprng to send jags call to multiple chains.
+++++++++++++++++++++++++++++++++

Some useful code using snow and R2WinBUGS package from Josh Nowak
I use the snow package with R2WinBUGS regularly.  I know you are using jags, but the code is basically the same.  There is no problem calling packages or your own functions using snowfall, they just have to be loaded on each processor, for example:
##########
require(snowfall)
sfInit(parallel = T, cpus = 4)
sfLibrary(R2WinBUGS)        #  Loading a library/package on all processors
sfSource("D:/code/model.run.r")  #  Loading a personal function on all processors, this contains my data generation, model and model call
#The last step is to call the function in some sort of loop, I tend to use the sfLapply function like so...
x <- sfLapply(1:4, function(i){
                   xx <- model.run( site = i )
    })
 sfStop()
##########

Now x is a list containing all of my results that were computed on four processors.  
+++++++++++++++++++++++++++++++++


Peter Solymos is the author of the R package dclone and he recommended:
Check out the jags.parfit() function in the dclone package, it runs parallel MCMC chains with the snow package. I will upload a bugfix release to CRAN soon, so if in any case you want to use parallel RNG
setup, you should get developmental version from http://dcr.r-forge.r-project.org . JAGS model inicialization is not affected by this issue, it can be important if you are using and RNG function for initial values, etc. Any feedback is welcome.
+++++++++++++++++++++++++++++++++

And Adam Wilson also pointed me at dclone:
I think this will be useful: http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=dclone:jags.parfit
+++++++++++++++++++++++++++++++++

And Guillaume Chapron replied on the possibilities of using the R package runjags with the Mac's xgrid cluster controller:

Perhaps try this

http://cran.r-project.org/web/packages/runjags/

If you do not have a server, set up xgrid on your Mac as both a controller and an agent so that you have a cluster with one machine, and then all the cores will be used.
+++++++++++++++++++++++++++++++++