[Bioc-devel] lpsymphony - BiocParallel crash

[Christian Arnold]

Dear Bioconductor developers,


I am having a somewhat mysterious and challenging problem which we 
believe is a bug in either (1) BiocParallel or (2) the lpsymphony 
library from either Bioconductor or the SYMPHONY backend.

The problem is, in a nutshell, that R silently crashes or, to be more 
precise, never finishes the calculation when using the only function 
from lpsymphony in combination with BiocParallel.

We updated the latest version of lpsymphony last week, did not help. 
This can be reproduced with two different configurations, using 
different library versions of both BiocParallel and lpsymphony, 
respectively. We also tested this on two different machines (one of 
which runs R in the devel version, one does not; see the two 
sessionInfo() at the end of this message)


The following code can reproduce the problem:


/library(BiocParallel)/

/library(lpsymphony)/


/lpsymphonyTest <- function(x){/

/# Example from the R help of lpsymphony_solve_LP/

/obj <- c(2, 4, 3)/

/mat <- matrix(c(3, 2, 1, 4, 1, 3, 2, 2, 2), nrow = 3)/

/dir <- c("<=", "<=", "<=")/

/rhs <- c(60, 40, 80)/

/max <- TRUE/

/test = lpsymphony_solve_LP(obj, mat, dir, rhs, max = max)/

/Sys.sleep(1)/

/return(1)/

/}/


/# First run: Two iterations, should run in parallel/

/nIterationsAndCores = 2/

/register(MulticoreParam(workers = nIterationsAndCores))/

/commonGenes <- bplapply(X = seq(2), FUN = lpsymphonyTest) /


/# Second run: One iteration, should run on only one core/

/commonGenes <- bplapply(X = seq(1), FUN = lpsymphonyTest) /


/# Third run: Two iterations again, should run in parallel but crashed 
and never finishes/

/commonGenes <- bplapply(X = seq(2), FUN = lpsymphonyTest) /


What happens in our case is that the third run never finishes. The two R 
processes are in state “SLEEP” rather than running. Once you execute the 
bplapply loop for only one iteration, everything after with number of 
iterations > 1 crashes. It works fine for any number of iterations > 1 
despite of the order, but again, executing only a single iteration 
causes the problems afterwards. Note that using registering again for 
the second and third run does not help and yields the same behavior.



In fact, we have another issue, namely that the parallelization with 
BiocParallel in combination with the IHW package (which calls lpsymphony 
in the background) does not yield running times as expected but instead 
much, much longer ones. However, let's focus on this one first, because 
we think they might be related.


Thanks for your help,


Best

Christian




These are the two configurations where this problem can be reproduced:


(1)

/R Under development (unstable) (2016-06-30 r70858)/

//

/Platform: x86_64-pc-linux-gnu (64-bit)/

//

/Running under: Ubuntu 16.04.1 LTS/

//

/
/

//

/locale:/

//

/[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=de_DE.UTF-8 
LC_COLLATE=en_US.UTF-8 /

//

/[5] LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=en_US.UTF-8 
LC_PAPER=de_DE.UTF-8 LC_NAME=C /

//

/[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=de_DE.UTF-8 
LC_IDENTIFICATION=C /

//

/attached base packages:/

//

/[1] stats graphics grDevices utils datasets methods base /

//

/other attached packages:/

//

/[1] lpsymphony_1.1.2 BiocParallel_1.7.8/

//

/loaded via a namespace (and not attached):/

//

/[1] parallel_3.4.0 tools_3.4.0 /



(2)

/R version 3.3.1 (2016-06-21)/

//

/Platform: x86_64-pc-linux-gnu (64-bit)/

//

/Running under: CentOS release 6.5 (Final)/

//

/
//locale:/

//

/[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 /

//

/[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 
LC_MESSAGES=en_US.UTF-8 /

//

/[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C /

//

/[10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C /

//

/
//attached base packages:/

//

/[1] stats graphics grDevices utils datasets methods base /

//

/
//other attached packages:/

//

/[1] lpsymphony_1.0.2 BiocParallel_1.6.6/

//

/
//loaded via a namespace (and not attached):/

//

/[1] parallel_3.3.1 tools_3.3.1 /


	[[alternative HTML version deleted]]