This is what I have right now. I think it can be improved to give print a
better summary when there are multiple unique error messages. The call to
castList is obviously not something that would go into a base package.

safe_mclapply <- function(X, FUN, ...) {
  ans <- IRanges:::castList(mclapply(X, function(...) {
    try(FUN(...), silent = !getOption("verbose"))
  }, ...))
  cond <- batchCondition(ans)
  if (!is.null(cond))
    stop(cond)
  ans
}

safe_mcmapply <- function(FUN, ...) {
  ans <- IRanges:::castList(mcmapply(function(...) {
    try(FUN(...), silent = !getOption("verbose"))
  }, ..., SIMPLIFY = FALSE))
  cond <- batchCondition(ans)
  if (!is.null(cond)) {
    stop(cond)
  }
  ans
}

batchCondition <- function(x) {
  errors <- sapply(x, is, "try-error")
  if (any(errors)) {
    mesg <- paste(sum(errors), "job(s) encountered an error.")
    error_mesg <- unlist(x[errors], use.names = FALSE)
    unique_mesg <- unique(error_mesg)
    if (length(unique_mesg) == 1L)
      mesg <- paste(mesg, "The error was the same for all nodes:",
unique_mesg)
    mesg <- paste(mesg,
                  "Call 'summary' or 'detail' on this condition object",
                  "for a report.")
    cond <- simpleError(mesg)
    cond$detail <- DataFrame(job = which(errors), error = error_mesg)
    class(cond) <- c("batchCondition", class(cond))
    cond
  } else NULL
}

setOldClass("batchCondition")

setMethod("summary", "batchCondition", function(object) {
  rename(as(xtabs(~ error, object$detail), "DataFrame"), Freq = "count")
})

setMethod("detail", "batchCondition", function(x) {
  x$detail
})

setMethod("show", "batchCondition", function(object) {
  cat(conditionMessage(object))
})


On Tue, Sep 25, 2012 at 8:43 AM, Cook, Malcolm <MEC@stowers.org> wrote:

>
> > -----Original Message-----
> > From: bioc-devel-bounces@r-project.org [mailto:
> bioc-devel-bounces@r-project.org] On Behalf Of Michael Lawrence
> > Sent: Monday, September 24, 2012 1:43 PM
> > To: Vincent Carey
> > Cc: Michael Lawrence; bioc-devel@r-project.org
> > Subject: Re: [Bioc-devel] mclapply and Vector objects
> >
> > Good points, Vince. Yes, it would be great to use the more general
> > parLapply and expect the client to somehow pass a cluster object that
> > controls the parallelization strategy. The mclapply function is a little
> > bit special in that it is able to inherit the enclosing environment from
> > the parent process. This is not generally feasible. So it seems OK to
> have
> > an mclapply wrapper that makes this assumption clear in a way that is
> > easier to read (in my opinion) from a parameterized apply function
> > accepting a closure, cluster object, or whatever.
> >
> > We actually don't have to do too much copying of the formals to the
> > wrapper, because we can simply help mclapply find our as.list generic:
> >
> > .mclapplyDefault <- parallel::mclapply
> > environment(.mclapplyDefault) <- topenv()
> > setMethod("mclapply", "List", .mclapplyDefault)
>
> Michael, I'm close to presenting something that i think will address all
> concerns, and am review this small flurry of emails to make sure.
>
> Would you please give me a quick explanation of that the above 'trick'
> accomplishes.  I'm still knocking around R's OO frameworks.
>
> Also...
>
> > Tricks aside, mclapply has some issues that would be nice to address. The
> > main one for me is error handling. I have developed some code that wraps
> > the user function in try() and compiles any errors into a special
> > "batchCondition" object that provides access to the individual exceptions
> > and gives a nice summary of what went wrong. Ideally, that would go into
> > parallel, but we could put it in IRanges as a stop gap.
>
> mclapply currently returns a vector of individual error messages as value
> if any process.  Are you suggesting perhaps mclapply should, in such cases,
> instead of warn, rather `stop` with a condition (batchCondition ),
> providing as additional attributes to the condition the vector conditions
> (try-errors) returned from each process.
>
> If so, yeah, yeah, I like.... sounds great.... but would be a change in
> behaviour to mclapply.... though probably an improvement that no-one would
> object to.
>
> What we have now, like this:
>
> x<-try(mclapply(1:5,simpleError('Hey!'),mc.silent=TRUE))
> Warning message:
> In mclapply(1:5, stop, simpleError("Hey!"), mc.silent = TRUE) :
>   all scheduled cores encountered errors in user code
> > class(x)
> [1] "list"
> > class(x[[1]])
> [1] "character"
> >x[[1]]
> [1] "Error in lapply(X=S, FUN=FUN,...) : 1 Error: I warned You\n\n"
>
>
>
> Instead, we are saying, mclapply should raise a batchError which would
> have as an attribute each individual core's simpleError, say, 'jobCondition'
>
> What could be simpler?
>
> It would then look like this:
>
> x<-try(mclapply(1:5,simpleError('Hey!'),mc.silent=TRUE))
> Error:
> In x<-try(mclapply(1:5,simpleError('Hey!'),mc.silent=TRUE))
>   all scheduled cores encountered errors in user code
> > class(x)
>  [1] "batchError" "simpleError" "error" "condition"
> > x
> [1] Error:
> In mclapply(1:5, stop, simpleError("Hey!"), mc.silent = TRUE) :
>   all scheduled cores encountered errors in user code
> attr(,"class")
> [1] "batchError" "simpleError" "error" "condition"
> attr(,"condition")
> <batchError  all scheduled cores encountered errors in user code>
> attr(,"jobCondition")
>                 [,1]    [,2]    [,3]    [,4]    [,5]
> jobid           1       2       3       4       5
> message Hey!    Hey!    Hey!    Hey!    Hey!
> call            NULL    NULL    NULL    NULL    NULL
>
>
> This would be sweet and simple.
>
>
> ~Malcolm
>
>
>
>
> >
> > Michael
> >
> > On Mon, Sep 24, 2012 at 7:12 AM, Vincent Carey
> > <stvjc@channing.harvard.edu>wrote:
> >
> > > only caveat is: do we want to commit to mc* in the interface or remain
> > > agnostic and allow
> > > iterator selection to be dropped in?
> > >
> > > i looked at the commented out mcseqapply and it seems unfortunate to
> > > manually propagate all the
> > > mc.* options
> > >
> > > so what am i suggesting?  i myself had to wonder.
> > >
> > > interactively, i generally use mclapply(1:N, function(ind) ...) to do
> get
> > > multicore processing for
> > > general objects, and when i want a higher-level function that allows
> users
> > > to choose for or against multicore
> > > iteration, define an applier parameter that defaults to lapply ... if
> you
> > > have to set options it is probably OK to do that through a closure, if
> you
> > > don't want to have all those potentially unstable parameters cluttering
> > > your arg list.
> > >
> > > so my proposal is: whatever we choose, plan for alternative approaches
> to
> > > multicore execution, and keep the code base slim by allowing the
> > > alternatives to be chosen through parameter settings as opposed to
> distinct
> > > interfaces
> > >
> > >
> > > On Mon, Sep 24, 2012 at 9:36 AM, Michael Lawrence <
> > > lawrence.michael@gene.com> wrote:
> > >
> > >>  I should amend this: it would be a method for the List class. Many
> of the
> > >> Vector classes are "atomic" and coercing them to a list is either not
> > >> supported or may yield an undesired result. For example, coercing an
> > >> IRanges to a list yields a list of integer vectors with the sequence
> from
> > >> start to end. We don't have an lapply,Vector for this reason.
> > >>
> > >> I actually already made a commented-out mcseqapply. I think I aborted
> the
> > >> mc* stuff back before the parallel package existed, just to avoid
> adding a
> > >> dependency on multicore. With parallel in base R, it's reasonable to
> add
> > >> these methods. If no one else complains, I'll move ahead.
> > >>
> > >> Michael
> > >>
> > >> On Mon, Sep 24, 2012 at 6:23 AM, Michael Lawrence <michafla@gene.com>
> > >> wrote:
> > >>
> > >> > It definitely makes sense to have a generic for mclapply that
> dispatches
> > >> > on Vector. Perhaps also for some of the other apply functions in the
> > >> > parallel package.
> > >> >
> > >> > Michael
> > >> >
> > >> >
> > >> > On Mon, Sep 24, 2012 at 3:58 AM, Hahne, Florian <
> > >> > florian.hahne@novartis.com> wrote:
> > >> >
> > >> >> Would it be possible to make mclapply aware of the Vector class?
> > >> >> Currently, the following line causes them to be coerced into a
> regular
> > >> >> list which could be rather expensive for instance in the case of
> > >> >> GrangesLists:
> > >> >> if (!is.vector(X) || is.object(X))
> > >> >>         X <- as.list(X)
> > >> >>
> > >> >>
> > >> >> I guess something like
> > >> >> if ((!is.vector(X) && !is(X, "Vector")) || is.object(X))
> > >> >>         X <- as.list(X)
> > >> >> would do the trick.
> > >> >>
> > >> >> Or am I missing something obvious here?
> > >> >> Cheers,
> > >> >> Florian
> > >> >>
> > >> >> _______________________________________________
> > >> >> Bioc-devel@r-project.org mailing list
> > >> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >> >>
> > >> >
> > >> >
> > >>
> > >>         [[alternative HTML version deleted]]
> > >>
> > >>
> > >> _______________________________________________
> > >> Bioc-devel@r-project.org mailing list
> > >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >>
> > >
> > >
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioc-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]

