[R-sig-hpc] foreach error checking (when a cluster stops)

[Stephen Weston]

Hi Jonathan,

Sorry, there's is nothing like that in foreach.  The functions that
are used to provide information about the parallel backend include:

  getDoParWorkers()
  getDoParName()
  getDoParVersion()
  getDoParRegistered()

but no "health check" functions.  It's a good idea, though.  I believe
they include that sort of thing in database frameworks.

By the way, I had no idea that you could use the doSNOW package
with snowfall, although I guess it makes sense since they give you
access to the cluster object.  But I've only ever tested doSNOW with
snow.

- Steve


On Sat, Feb 2, 2013 at 12:19 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
> R-sig-hpc'ers:
>
> I've been modding some code lately to take advantage of the foreach
> generalization (so I don't have to write multiple statements depending on
> the parallel backend that I have running), but I'm trying to figure out how
> to check for a registered, but no longer running, parallel backend, e.g.:
>
> require("foreach")
> require("snowfall")
> require("doSNOW")
> # Start a snowfall cluster and register it with foreach
> sfInit(cpu=2,parallel=TRUE)
> cl <- sfGetCluster()
> registerDoSNOW(cl)
>
> # Some foreach statement:
> d <- data.frame(x=1:10, y=rnorm(10))
> s <- foreach(d=iter(d, by='row'), .combine=rbind) %dopar% d
>
> # Works fine.  Now stop the cluster:
> sfStop()
> s <- foreach(d=iter(d, by='row'), .combine=rbind) %dopar% d
> # Doesn't work, because the snow cluster is gone.
>
> # I can manually set this, but is there some test I can do
> # to test for whether the cluster is working and, if not:
> registerDoSEQ()
> s <- foreach(d=iter(d, by='row'), .combine=rbind) %dopar% d
>
> # e.g.
> if(checkForWorkingBackend == FALSE)
> {
> registerDoSEQ()
> }
>
>
>
> --
> Jonathan A. Greenberg, PhD
> Assistant Professor
> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
> Department of Geography and Geographic Information Science
> University of Illinois at Urbana-Champaign
> 607 South Mathews Avenue, MC 150
> Urbana, IL 61801
> Phone: 217-300-1924
> http://www.geog.illinois.edu/~jgrn/
> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc