Good points, Vince. Yes, it would be great to use the more general
parLapply and expect the client to somehow pass a cluster object that
controls the parallelization strategy. The mclapply function is a little
bit special in that it is able to inherit the enclosing environment from
the parent process. This is not generally feasible. So it seems OK to have
an mclapply wrapper that makes this assumption clear in a way that is
easier to read (in my opinion) from a parameterized apply function
accepting a closure, cluster object, or whatever.

We actually don't have to do too much copying of the formals to the
wrapper, because we can simply help mclapply find our as.list generic:

.mclapplyDefault <- parallel::mclapply
environment(.mclapplyDefault) <- topenv()
setMethod("mclapply", "List", .mclapplyDefault)

Tricks aside, mclapply has some issues that would be nice to address. The
main one for me is error handling. I have developed some code that wraps
the user function in try() and compiles any errors into a special
"batchCondition" object that provides access to the individual exceptions
and gives a nice summary of what went wrong. Ideally, that would go into
parallel, but we could put it in IRanges as a stop gap.

Michael

On Mon, Sep 24, 2012 at 7:12 AM, Vincent Carey
<stvjc@channing.harvard.edu>wrote:

> only caveat is: do we want to commit to mc* in the interface or remain
> agnostic and allow
> iterator selection to be dropped in?
>
> i looked at the commented out mcseqapply and it seems unfortunate to
> manually propagate all the
> mc.* options
>
> so what am i suggesting?  i myself had to wonder.
>
> interactively, i generally use mclapply(1:N, function(ind) ...) to do get
> multicore processing for
> general objects, and when i want a higher-level function that allows users
> to choose for or against multicore
> iteration, define an applier parameter that defaults to lapply ... if you
> have to set options it is probably OK to do that through a closure, if you
> don't want to have all those potentially unstable parameters cluttering
> your arg list.
>
> so my proposal is: whatever we choose, plan for alternative approaches to
> multicore execution, and keep the code base slim by allowing the
> alternatives to be chosen through parameter settings as opposed to distinct
> interfaces
>
>
> On Mon, Sep 24, 2012 at 9:36 AM, Michael Lawrence <
> lawrence.michael@gene.com> wrote:
>
>>  I should amend this: it would be a method for the List class. Many of the
>> Vector classes are "atomic" and coercing them to a list is either not
>> supported or may yield an undesired result. For example, coercing an
>> IRanges to a list yields a list of integer vectors with the sequence from
>> start to end. We don't have an lapply,Vector for this reason.
>>
>> I actually already made a commented-out mcseqapply. I think I aborted the
>> mc* stuff back before the parallel package existed, just to avoid adding a
>> dependency on multicore. With parallel in base R, it's reasonable to add
>> these methods. If no one else complains, I'll move ahead.
>>
>> Michael
>>
>> On Mon, Sep 24, 2012 at 6:23 AM, Michael Lawrence <michafla@gene.com>
>> wrote:
>>
>> > It definitely makes sense to have a generic for mclapply that dispatches
>> > on Vector. Perhaps also for some of the other apply functions in the
>> > parallel package.
>> >
>> > Michael
>> >
>> >
>> > On Mon, Sep 24, 2012 at 3:58 AM, Hahne, Florian <
>> > florian.hahne@novartis.com> wrote:
>> >
>> >> Would it be possible to make mclapply aware of the Vector class?
>> >> Currently, the following line causes them to be coerced into a regular
>> >> list which could be rather expensive for instance in the case of
>> >> GrangesLists:
>> >> if (!is.vector(X) || is.object(X))
>> >>         X <- as.list(X)
>> >>
>> >>
>> >> I guess something like
>> >> if ((!is.vector(X) && !is(X, "Vector")) || is.object(X))
>> >>         X <- as.list(X)
>> >> would do the trick.
>> >>
>> >> Or am I missing something obvious here?
>> >> Cheers,
>> >> Florian
>> >>
>> >> _______________________________________________
>> >> Bioc-devel@r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>
>> >
>> >
>>
>>         [[alternative HTML version deleted]]
>>
>>
>> _______________________________________________
>> Bioc-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>
>

	[[alternative HTML version deleted]]

