[R] Problem with mclapply -- losing output/data
Elizabeth Purdom
epurdom at stat.berkeley.edu
Tue Mar 22 09:13:40 CET 2011
Hello,
I am running large simulations, which unfortunately I can't really
replicate here because the code is so extensive. I rely heavily on
mclapply, but I realize that I'm losing data somewhere.
There are two worrisome symptoms:
1) I am getting 'NULL' as a return value for some (but not all) elements
of the output when I use mclapply, but not if I use lapply
> tmp2[1:3] #output from lapply
[[1]]
10000076 10000077
24 24
[[2]]
10000076 10000077
119 119
[[3]]
10000076
71
> tmp[1:3] #output from mclapply
[[1]]
NULL
[[2]]
NULL
[[3]]
NULL
2) I am not getting back a list the same length as my input vector I'm
parallelizing over. i.e. a command like this:
tmp<-mclapply(x, FUN=myfunc, mc.cores=16)
gives me back a list tmp which is not the same length as x (and so I'm
getting all kinds of errors)
This is extremely discouraging, because I've been using mclapply
extensively at very many points on simulations that take a very long
time to run, and now I'm wondering if what I'm getting is trustworthy. I
don't think I could reasonably finish my results without mclapply, but I
am thinking to cut it out except where it was absolutely necessary,
time-wise. If anyone had any suggestions as to why this might be
happening and how I can circumvent it (or test for it happening), I
would greatly appreciate it.
Thanks,
Elizabeth Purdom
> sessionInfo()
R version 2.12.1 (2010-12-16)
Platform: x86_64-pc-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8
LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] multicore_0.1-4 msm_1.0 gtools_2.6.2
graph_1.28.0 Rsamtools_1.2.3
[6] Biostrings_2.18.2 GenomicFeatures_1.2.3 GenomicRanges_1.2.3
IRanges_1.8.9
loaded via a namespace (and not attached):
[1] Biobase_2.10.0 biomaRt_2.6.0 BSgenome_1.18.3
DBI_0.2-5 mvtnorm_0.9-96 RCurl_1.5-0
[7] RSQLite_0.9-4 rtracklayer_1.10.6 splines_2.12.1
survival_2.36-2 tools_2.12.1 XML_3.2-0
More information about the R-help
mailing list