[Bioc-devel] Error: node stack overflow
Hervé Pagès
hpages at fredhutch.org
Tue May 1 01:48:08 CEST 2018
Great, thanks!
I was curious and did:
hpages at spectre:~/svn/R/R-3-5-branch$ svn diff -r 74669:74674
src/library/methods/R/methodsTable.R
Index: src/library/methods/R/methodsTable.R
===================================================================
--- src/library/methods/R/methodsTable.R (revision 74669)
+++ src/library/methods/R/methodsTable.R (revision 74674)
@@ -697,7 +697,7 @@
.findNextFromTable <- function(method, f, optional, envir, prev =
character())
{
- fdef <- getGeneric(f)
+ fdef <- getGeneric(f, where=envir)
env <- environment(fdef)
## target <- method at target
n <- get(".SigLength", envir = env)
That was it? Whao! Probably one of the best illustration I've seen
that a hard-to-reproduce bug doesn't necessarily require a complex
or sophisticated fix ;-) I'm not implying that the fix was easy here,
a proper fix can be hard to figure out even if it's simple.
BTW, before the fix, the 'envir' argument was ignored so I wonder if
a static code analysis tool couldn't have detected this...
H.
On 04/30/2018 03:26 PM, Michael Lawrence wrote:
> I just pushed it to the 3.5 branch.
>
> On Mon, Apr 30, 2018 at 2:14 PM, Hervé Pagès <hpages at fredhutch.org> wrote:
>> Excellent! Are you planning to commit this to the 3.5 branch too?
>> In that case we'll wait a couple more days before installing R 3.5
>> patched on the build machines for the BioC 3.8 builds.
>>
>> Thanks,
>> H.
>>
>>
>> On 04/30/2018 01:43 PM, Michael Lawrence wrote:
>>>
>>> It's checked into devel now. Thanks for the well documented examples,
>>> Hervé.
>>>
>>> On Mon, Apr 30, 2018 at 10:26 AM, Michael Lawrence <michafla at gene.com>
>>> wrote:
>>>>
>>>> I've fixed it and will push to R-devel as soon as it passes checks.
>>>>
>>>> Michael
>>>>
>>>> On Sun, Apr 29, 2018 at 9:04 PM, Michael Lawrence <michafla at gene.com>
>>>> wrote:
>>>>>
>>>>> Just noticed this thread. I will look into this and hopefully fix it.
>>>>>
>>>>> On Sun, Apr 29, 2018 at 6:12 PM, Hervé Pagès <hpages at fredhutch.org>
>>>>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I made progress on this. This has actually nothing to do with Java.
>>>>>> You get the same thing with the flexmix package. What rJava and flexmix
>>>>>> have in common is that they both define a method on the base::unique()
>>>>>> implicit S4 generic.
>>>>>>
>>>>>> The issue actually originates in the methods package. In order to
>>>>>> remove
>>>>>> rJava, BiocGenerics and IRanges from the equation, I made 2 minimalist
>>>>>> packages, uniqueMethod and uniqueGeneric, that can be used to reproduce
>>>>>> the issue. See:
>>>>>>
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Bioconductor_uniqueGeneric&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=hI6LWw_u7csv2YoouDZz4PmV3GnTtE0movmB_pZuog8&s=5UWyuXnFrW3P7eeuT0jq8O9gi7BOreYKUmZ0LdDpJ3M&e=
>>>>>>
>>>>>>
>>>>>> I committed a workaround in S4Vectors (0.17.44). With this version of
>>>>>> S4Vectors:
>>>>>>
>>>>>> library(rJava)
>>>>>> library(IRanges)
>>>>>> unique(IRanges())
>>>>>> # IRanges object with 0 ranges and 0 metadata columns:
>>>>>> # start end width
>>>>>> # <integer> <integer> <integer>
>>>>>>
>>>>>> Let me know if you still run into problems with this.
>>>>>>
>>>>>> Cheers,
>>>>>> H.
>>>>>>
>>>>>>> sessionInfo()
>>>>>>
>>>>>> R Under development (unstable) (2018-02-26 r74306)
>>>>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>>>> Running under: Ubuntu 16.04.4 LTS
>>>>>>
>>>>>> Matrix products: default
>>>>>> BLAS: /home/hpages/R/R-3.5.r74306/lib/libRblas.so
>>>>>> LAPACK: /home/hpages/R/R-3.5.r74306/lib/libRlapack.so
>>>>>>
>>>>>> locale:
>>>>>> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>>>>>> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>>>>>> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>>>>>> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
>>>>>> [9] LC_ADDRESS=C LC_TELEPHONE=C
>>>>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>>>>>
>>>>>> attached base packages:
>>>>>> [1] stats4 parallel stats graphics grDevices utils
>>>>>> datasets
>>>>>> [8] methods base
>>>>>>
>>>>>> other attached packages:
>>>>>> [1] IRanges_2.13.29 S4Vectors_0.17.44 BiocGenerics_0.25.3
>>>>>> [4] rJava_0.9-9
>>>>>>
>>>>>> loaded via a namespace (and not attached):
>>>>>> [1] compiler_3.5.0
>>>>>>
>>>>>>
>>>>>> On 04/14/2018 03:11 AM, Hervé Pagès wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi Zheng,
>>>>>>>
>>>>>>> I can totally reproduce this on my Ubuntu laptop:
>>>>>>>
>>>>>>> library(rJava)
>>>>>>> library(IRanges)
>>>>>>> unique(IRanges())
>>>>>>> # Error in validObject(.Object) :
>>>>>>> # invalid class “MethodWithNext” object: Error : C stack usage
>>>>>>> 7969396 is too close to the limit
>>>>>>>
>>>>>>> See my seesionInfo() at the end of this email.
>>>>>>>
>>>>>>> Probably related to this (but not 100% sure) loading rJava seems
>>>>>>> to break selectMethod().
>>>>>>>
>>>>>>> More precisely: The rJava package defines some "unique" S4 methods
>>>>>>> and the BiocGenerics package defines (and exports) the unique() S4
>>>>>>> generic with the following statement:
>>>>>>>
>>>>>>> setGeneric("unique", signature="x")
>>>>>>>
>>>>>>> Here is what happens when loading the rJava package first:
>>>>>>>
>>>>>>> library(rJava)
>>>>>>> library(BiocGenerics)
>>>>>>>
>>>>>>> setClass("A", slots=c(a="integer"))
>>>>>>> setMethod("unique", "A",
>>>>>>> function(x, incomparables=FALSE, ...) {x at a <- unique(x at a); x}
>>>>>>> )
>>>>>>>
>>>>>>> selectMethod("unique", "A")
>>>>>>> # Method Definition (Class "derivedDefaultMethod"):
>>>>>>> #
>>>>>>> # function (x, incomparables = FALSE, ...)
>>>>>>> # UseMethod("unique")
>>>>>>> # <bytecode: 0x4127140>
>>>>>>> # <environment: namespace:base>
>>>>>>> #
>>>>>>> # Signatures:
>>>>>>> # x
>>>>>>> # target "A"
>>>>>>> # defined "ANY"
>>>>>>>
>>>>>>> selectMethod() doesn't find the method for A objects!
>>>>>>>
>>>>>>> It seems that selectMethod() is looking in the method table for
>>>>>>> the implicit unique() generic defined in rJava instead of the
>>>>>>> explicit unique() generic defined in BiocGenerics. If we tell
>>>>>>> selectMethod() which generic to consider, then it finds the method
>>>>>>> for A objects:
>>>>>>>
>>>>>>> selectMethod(BiocGenerics::unique, "A")
>>>>>>> # Method Definition:
>>>>>>> #
>>>>>>> # function (x, incomparables = FALSE, ...)
>>>>>>> # {
>>>>>>> # x at a <- unique(x at a)
>>>>>>> # x
>>>>>>> # }
>>>>>>> #
>>>>>>> # Signatures:
>>>>>>> # x
>>>>>>> # target "A"
>>>>>>> # defined "A"
>>>>>>>
>>>>>>> In order to reproduce the above problem without the BiocGenerics
>>>>>>> package in the equation, it's not enough to do:
>>>>>>>
>>>>>>> library(rJava)
>>>>>>> setGeneric("unique", signature="x")
>>>>>>> etc...
>>>>>>>
>>>>>>> The setGeneric("unique", signature="x") statement must be put in
>>>>>>> a package. I've created a minimalist package on GitHub that just
>>>>>>> wraps this statement:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Bioconductor_uniqueGeneric&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=b4YM1TCcJjPge2siQJS5BQv7g1DMkoCQ-7FvZz89w-E&s=P9-WHLV4FmY9PQcNjkZ4Cgc24Oi1QTNVBqTaQ1iS-kg&e=
>>>>>>>
>>>>>>> This package can be used instead of BiocGenerics to reproduce the
>>>>>>> problem above.
>>>>>>>
>>>>>>> I'm not 100% sure that this problem is related to the issue you
>>>>>>> reported originally but it seems very likely to me.
>>>>>>>
>>>>>>> Not quite sure what the next step should be. I've been told by
>>>>>>> some R core developers that there are known interaction issues
>>>>>>> between Java, rJava and R that are currently being worked on.
>>>>>>> Someone should ask on the R-devel mailing list or directly to
>>>>>>> Simon Urbanek, the rJava author, for more information about this.
>>>>>>>
>>>>>>> H.
>>>>>>>
>>>>>>> > sessionInfo()
>>>>>>> R Under development (unstable) (2018-02-26 r74306)
>>>>>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>>>>> Running under: Ubuntu 16.04.4 LTS
>>>>>>>
>>>>>>> Matrix products: default
>>>>>>> BLAS: /home/hpages/R/R-3.5.r74306/lib/libRblas.so
>>>>>>> LAPACK: /home/hpages/R/R-3.5.r74306/lib/libRlapack.so
>>>>>>>
>>>>>>> locale:
>>>>>>> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>>>>>>> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>>>>>>> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>>>>>>> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
>>>>>>> [9] LC_ADDRESS=C LC_TELEPHONE=C
>>>>>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>>>>>>
>>>>>>> attached base packages:
>>>>>>> [1] stats graphics grDevices utils datasets methods base
>>>>>>>
>>>>>>> other attached packages:
>>>>>>> [1] BiocGenerics_0.25.3 rJava_0.9-9
>>>>>>>
>>>>>>> loaded via a namespace (and not attached):
>>>>>>> [1] compiler_3.5.0 parallel_3.5.0
>>>>>>>
>>>>>>>
>>>>>>> On 04/02/2018 11:47 AM, Vincent Carey wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> I get the following on MacOSX, after library(rJava);
>>>>>>>> library(GenomicRanges) -- take rJava out of the mix and
>>>>>>>> there is no error. I ran into this originally as an interaction
>>>>>>>> between
>>>>>>>> rJava and Gviz, and have reported to
>>>>>>>> Florian. That seemed to be MacOSX-specific.
>>>>>>>>
>>>>>>>>> gr1 <- GRanges(seqnames=Rle(c("ch1", "chMT"), c(2, 4)),
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> +ranges=IRanges(16:21, 20),
>>>>>>>>
>>>>>>>> +strand=rep(c("+", "-", "*"), 2))
>>>>>>>>
>>>>>>>>> unique(gr1)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> *Error in validObject(.Object) :*
>>>>>>>>
>>>>>>>> *invalid class "MethodWithNext" object: Error : C stack usage7969864
>>>>>>>> is
>>>>>>>> too close to the limit*
>>>>>>>>
>>>>>>>> *Error during wrapup: C stack usage7969656 is too close to the limit*
>>>>>>>>
>>>>>>>> *
>>>>>>>> *
>>>>>>>>
>>>>>>>> **
>>>>>>>>
>>>>>>>> *> sessionInfo()*
>>>>>>>>
>>>>>>>> *
>>>>>>>>
>>>>>>>> R Under development (unstable) (2018-02-14 r74250)
>>>>>>>>
>>>>>>>> Platform: x86_64-apple-darwin15.6.0 (64-bit)
>>>>>>>>
>>>>>>>> Running under: macOS Sierra 10.12.6
>>>>>>>>
>>>>>>>>
>>>>>>>> Matrix products: default
>>>>>>>>
>>>>>>>> BLAS:
>>>>>>>>
>>>>>>>> /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
>>>>>>>>
>>>>>>>> LAPACK:
>>>>>>>>
>>>>>>>> /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
>>>>>>>>
>>>>>>>>
>>>>>>>> locale:
>>>>>>>>
>>>>>>>> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>>>>>>>>
>>>>>>>>
>>>>>>>> attached base packages:
>>>>>>>>
>>>>>>>> [1] parallelstats4stats graphicsgrDevices utils datasets
>>>>>>>>
>>>>>>>> [8] methods base
>>>>>>>>
>>>>>>>>
>>>>>>>> other attached packages:
>>>>>>>>
>>>>>>>> [1] GenomicRanges_1.31.23 GenomeInfoDb_1.15.5 IRanges_2.13.28
>>>>>>>>
>>>>>>>> [4] S4Vectors_0.17.38 BiocGenerics_0.25.3 rJava_0.9-9
>>>>>>>>
>>>>>>>> [7] rmarkdown_1.9
>>>>>>>>
>>>>>>>>
>>>>>>>> loaded via a namespace (and not attached):
>>>>>>>>
>>>>>>>> [1] Rcpp_0.12.16 digest_0.6.15rprojroot_1.3-2
>>>>>>>>
>>>>>>>> [4] bitops_1.0-6 backports_1.1.2magrittr_1.5
>>>>>>>>
>>>>>>>> [7] evaluate_0.10.1zlibbioc_1.25.0stringi_1.1.7
>>>>>>>>
>>>>>>>> [10] XVector_0.19.9 tools_3.5.0stringr_1.3.0
>>>>>>>>
>>>>>>>> [13] RCurl_1.95-4.10compiler_3.5.0 htmltools_0.3.6
>>>>>>>>
>>>>>>>> [16] knitr_1.20 GenomeInfoDbData_1.1.0
>>>>>>>>
>>>>>>>>
>>>>>>>> *
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 2, 2018 at 2:25 PM, Hervé Pagès <hpages at fredhutch.org
>>>>>>>> <mailto:hpages at fredhutch.org>> wrote:
>>>>>>>>
>>>>>>>> Hi Zheng,
>>>>>>>>
>>>>>>>> Thanks for the report. I will look into this and will let you
>>>>>>>> know.
>>>>>>>>
>>>>>>>> H.
>>>>>>>>
>>>>>>>> On 04/01/2018 02:38 AM, Zheng Wei wrote:
>>>>>>>>
>>>>>>>> Dear all,
>>>>>>>>
>>>>>>>> I find this error if calling library(rJava) before using
>>>>>>>> BiocGenerics::unique
>>>>>>>>
>>>>>>>> The code is pasted below.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Zheng
>>>>>>>>
>>>>>>>> > library(rJava)
>>>>>>>> > library(GenomicRanges)
>>>>>>>> Loading required package: stats4
>>>>>>>> Loading required package: BiocGenerics
>>>>>>>> Loading required package: parallel
>>>>>>>>
>>>>>>>> Attaching package: ‘BiocGenerics’
>>>>>>>>
>>>>>>>> The following objects are masked from ‘package:parallel’:
>>>>>>>>
>>>>>>>> clusterApply, clusterApplyLB, clusterCall,
>>>>>>>> clusterEvalQ
>>>>>>>> clusterExport, clusterMap, parApply, parCapply,
>>>>>>>> parLapp
>>>>>>>> parLapplyLB, parRapply, parSapply, parSapplyLB
>>>>>>>>
>>>>>>>> The following objects are masked from ‘package:rJava’:
>>>>>>>>
>>>>>>>> anyDuplicated, duplicated, sort, unique
>>>>>>>>
>>>>>>>> The following objects are masked from ‘package:stats’:
>>>>>>>>
>>>>>>>> IQR, mad, sd, var, xtabs
>>>>>>>>
>>>>>>>> The following objects are masked from ‘package:base’:
>>>>>>>>
>>>>>>>> anyDuplicated, append, as.data.frame, basename, cbind,
>>>>>>>> colnames, colSums, dirname, do.call, duplicated, eval,
>>>>>>>> Filter, Find, get, grep, grepl, intersect,
>>>>>>>> is.unsorted,
>>>>>>>> lengths, Map, mapply, match, mget, order, paste, pmax,
>>>>>>>> pmin, pmin.int
>>>>>>>>
>>>>>>>>
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__pmin.int&d=DwMFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=WuplCAfYBn5Cq3N946mtZUi0612IchE0DfGaLDRFWTg&s=Km1Jhe0uxvMMuNveRSNtoTyFaOBZRO2jL8kmCFQd8Ec&e=>,
>>>>>>>> Position, rank, rbind, Reduce, rowMeans
>>>>>>>> rowSums, sapply, setdiff, sort, table, tapply, union,
>>>>>>>> u
>>>>>>>> unsplit, which, which.max, which.min
>>>>>>>>
>>>>>>>> Loading required package: S4Vectors
>>>>>>>>
>>>>>>>>
>>>>>>>> Attaching package: ‘S4Vectors’
>>>>>>>>
>>>>>>>> The following object is masked from ‘package:base’:
>>>>>>>>
>>>>>>>> expand.grid
>>>>>>>>
>>>>>>>> Loading required package: IRanges
>>>>>>>> Loading required package: GenomeInfoDb
>>>>>>>> > gr1 <- GRanges(seqnames=Rle(c("ch1", "chMT"), c(2, 4)),
>>>>>>>> + ranges=IRanges(16:21, 20),
>>>>>>>> + strand=rep(c("+", "-", "*"), 2))
>>>>>>>> > unique(gr1)
>>>>>>>> Error: node stack overflow
>>>>>>>> > BiocGenerics::unique(gr1)
>>>>>>>> Error: node stack overflow
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -- Hervé Pagès
>>>>>>>>
>>>>>>>> Program in Computational Biology
>>>>>>>> Division of Public Health Sciences
>>>>>>>> Fred Hutchinson Cancer Research Center
>>>>>>>> 1100 Fairview Ave. N, M1-B514
>>>>>>>> P.O. Box 19024
>>>>>>>> Seattle, WA 98109-1024
>>>>>>>>
>>>>>>>> E-mail: hpages at fredhutch.org <mailto:hpages at fredhutch.org>
>>>>>>>> Phone: (206) 667-5791 <tel:%28206%29%20667-5791>
>>>>>>>> Fax: (206) 667-1319 <tel:%28206%29%20667-1319>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Bioc-devel at r-project.org <mailto:Bioc-devel at r-project.org>
>>>>>>>> mailing
>>>>>>>> list
>>>>>>>>
>>>>>>>>
>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=b4YM1TCcJjPge2siQJS5BQv7g1DMkoCQ-7FvZz89w-E&s=JNrsjIF-swIGjCJUiljVcp0KYi42-z3NNryBT3EZmSg&e=
>>>>>>>>
>>>>>>>>
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwMFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=WuplCAfYBn5Cq3N946mtZUi0612IchE0DfGaLDRFWTg&s=Lw0PlsxUz2dbA2mDa_e4vZrNZ2FfjaNNQ-abMHuIMNY&e=>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Hervé Pagès
>>>>>>
>>>>>> Program in Computational Biology
>>>>>> Division of Public Health Sciences
>>>>>> Fred Hutchinson Cancer Research Center
>>>>>> 1100 Fairview Ave. N, M1-B514
>>>>>> P.O. Box 19024
>>>>>> Seattle, WA 98109-1024
>>>>>>
>>>>>> E-mail: hpages at fredhutch.org
>>>>>> Phone: (206) 667-5791
>>>>>> Fax: (206) 667-1319
>>>>>>
>>>>>> _______________________________________________
>>>>>> Bioc-devel at r-project.org mailing list
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=hI6LWw_u7csv2YoouDZz4PmV3GnTtE0movmB_pZuog8&s=NJsCdpN9A1bsrpPUJSfAS8wTz1sXycGRPlBKr4-2M0E&e=
>>>>>>
>>
>> --
>> Hervé Pagès
>>
>> Program in Computational Biology
>> Division of Public Health Sciences
>> Fred Hutchinson Cancer Research Center
>> 1100 Fairview Ave. N, M1-B514
>> P.O. Box 19024
>> Seattle, WA 98109-1024
>>
>> E-mail: hpages at fredhutch.org
>> Phone: (206) 667-5791
>> Fax: (206) 667-1319
>>
--
Hervé Pagès
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpages at fredhutch.org
Phone: (206) 667-5791
Fax: (206) 667-1319
More information about the Bioc-devel
mailing list