[R-sig-hpc] doMC - Using the foreach function

[Stephen Weston]

Hi Fong,

The quick answer is to set the "names" attribute of the list returned
by "foreach":

    names(p_vals) <- as.character(ni[,2])

However, you could also set the "names" attribute of the values
returned by the "foreach" loop, and combine them using the
standard "c" function.

An important note is that you can't use either "next" or "break"
inside a "foreach" loop.  I believe that would require a change
to the R interpreter.  However, you can use the ".errorhandling"
argument to "foreach" to filter values out.

Here's my version of your example.  Note that I'm using an "icount"
iterator, rather than a vector to specify the values for "r".  That isn't
necessary, but I like to use iterators when appropriate.

   p_vals <- foreach(r=icount(nrow(ni)), .errorhandling='remove',
.combine='c') %dopar% {
      t_test_results <- t.test(ni[r, groupA], ni[r, groupB])
      if (is.nan(t_test_results$p.value)) stop('remove me')
      structure(t_test_results$p.value, names=as.character(ni[r, 2]))
   }

If "t.test" throws an error, or if p.value is a NaN, then no value is returned
for that iteration.  That seems to be the behavior that you're trying to
achieve.

Make sure that you use the latest version of the "doMC" package
on CRAN, since that contains a fix for a bug when the "foreach" body
throws an error.

Also note that this version returns a vector, not a list.  That seems to make
sense, but if you really want a list, just wrap t_test_results$p.value
in a list before passing it to "structure", as follows:

   p_vals <- foreach(r=icount(nrow(ni)), .errorhandling='remove',
.combine='c') %dopar% {
      t_test_results <- t.test(ni[r, groupA], ni[r, groupB])
      if (is.nan(t_test_results$p.value)) stop('remove me')
      structure(list(t_test_results$p.value), names=as.character(ni[r, 2]))
   }

It still uses "c" as the combine function, since "c" works on both
vectors and lists.

Finally, I'm not sure why you think that the ".combine" argument
doesn't work with custom functions.  I've used it many times without
problems.  There are also examples of custom combine functions that
come with the "foreach" package.  If you have an example that
demonstrates a problem, I'd be glad to take a look at it.  You could also
send a report to the package maintainer, but I'd be personally interested
in hearing of any bugs in "foreach" or "doMC".

- Steve



On Tue, Aug 3, 2010 at 6:51 PM, Fong Chun Chan <fongchunchan at gmail.com> wrote:
> Hi all,
>
> I just discovered the doMC package and have been working on parallelizing my
> code.  I know by default the foreach function returns a list in which the
> tags are the index of the iterator.  If I wanted specific tags for my list
> how could I do something like that?  For example, the following code does a
> t-test and returns a p-val for each probeset.
>
> p_vals <- NULL
> p_vals <- foreach (r = 1:nrow(ni)) %dopar% {
>    t_test_results <- try(t.test(ni[r, groupA], ni[r, groupB]), silent=TRUE)
>    ps <- as.character(ni[r,2])
>
>    if (is(t_test_results, "try-error")){
>        next()
>    } else {
>        if (!is.nan(t_test_results$p.value)){
>            print(paste("Performing t-test on probeset -", ps))
>            t_test_results$p.value
>        }
>    }
> }
>
> The returned list would look like:
>
> [1] 0.005
> [2] 0.15
> [3] 0.52
> .
> .
> .
>
> The list tag for the p-val is the index r.  But the probeset itself as a
> specific value (ps) which I would like to use as the tag in the list.  So it
> could be like this:
>
> [2425915] 0.005
> [1596109] 0.15
> [3258010] 0.52
> .
> .
> .
>
> I tried to use the line p_vals[[ps]] <- t_test_results$p.value inside my
> foreach loop but it doesn't work.  The .combine parameter doesn't appear to
> allow me to specify custom functions to combine my results.  Any ideas on
> what I could do?
>
> Thanks,
>
> Fong
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>