[R-sig-hpc] New R packages BatchJobs and BatchExperiments for computing on HPCs
simon.urbanek at r-project.org
Wed May 30 16:50:19 CEST 2012
would you like to give Rserve clusters a shot? They offer state, pre-loading, parallelization and authentication (in particular the new version including multi-user) with very efficient R dispatch - much better than any generic systems. I was going to add something like a parallel/show class for them, but what you describe goes even further ..
On May 30, 2012, at 9:22 AM, Michel Lang wrote:
> The packages BatchJobs and BatchExperiments are now available on CRAN.
> We believe some subscribers of this list may find the packages useful.
> Here is an excerpt of the abstract of our publication:
> The package BatchJobs implements the basic objects and procedures to
> control a batch cluster within R. It is structured around cluster
> versions of the well-known higher order functions Map, Reduce and
> Filter from functional programming. An important feature is that the
> state of computation is persistently available in a database. The user
> can query the status of jobs and then continue working with a desired
> The second package, BatchExperiments, is tailored for the still very
> general scenario of analyzing arbitrary algorithms on problem
> instances. It extends BatchJobs by letting the user define an array of
> jobs of the kind "apply algorithm A to problem instance P and store
> results". It is possible to associate statistical designs with
> parameters of algorithms and problems and therefore systematically
> study their influence on the results.
> We currently support the following batch systems:
> * TORQUE/PBS based systems
> * LSF
> * Sun/Oracle Grid Engine (SGE)
> * Makeshift SSH clusters to operate on loosely connected boxes with a
> shared common file system
> * Local interactive execution, mainly for prototyping
> * Local multicore execution
> Note that the adoption of other batch systems is in principle easily
> achieved, we simply had no access to them. If you want to see support
> for your batch system, please send us a mail.
> Fore more information, see
> * Project site: <http://batchjobs.googlecode.com>
> * Technical report on the packages, also submitted to JSS:
> * BatchJobs CRAN page:
> * BatchExperiments CRAN page:
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
More information about the R-sig-hpc