[Bioc-devel] bplapply Processes Sometimes Stall

Dario Strbenac dstr7320 at uni.sydney.edu.au
Mon Dec 28 02:00:07 CET 2015

Good day,

I have a minimal example of bplapply stalling.

results <- lapply(1:20, function(variety)
  message("Variety", variety)
  bplapply(1:100, function(index) {res <- list(sample(20000), sample(c("Healthy", "Disease"), 20000, replace = TRUE)); res}, BPPARAM = MulticoreParam(workers = 25))

It sometimes stalls on no particular iteration, but other times it runs all 20 iterations and returns to the R command prompt. It's not reproducible when the stall happens. I am trying to find the cause of a cross-validation loop that progresses for a few hours, then stalls. When the stall happens, two or three of the R processes appear to be using 100% CPU whereas the others are finished, according to the output of top. The server was previously running R 3.1.2 and Debian 7 and this didn't ever happen. The server has 48 processors.

If I set workers to 5, it always completes the loop and returns to the prompt. Using mclapply with mc.cores set to 25 also always works, so the problem is with bplapply.

R version 3.2.3 (2015-12-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 8 (jessie)

 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8        LC_COLLATE=C.UTF-8    
 [5] LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8    LC_PAPER=C.UTF-8       LC_NAME=C             

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] BiocParallel_1.4.3

loaded via a namespace (and not attached):
[1] futile.logger_1.4.1  lambda.r_1.1.7       futile.options_1.0.0

Dario Strbenac
PhD Student
University of Sydney
Camperdown NSW 2050

More information about the Bioc-devel mailing list