[Rd] [R] debug vs regular mode
Simon Urbanek
simon.urbanek at r-project.org
Sat Aug 11 03:45:17 CEST 2012
On Aug 10, 2012, at 7:08 PM, Zhang, Peng wrote:
> Hi Simon,
>
> That could be the explanation. Is it possible to know stack limit for
> different scenarios?
The limit is the same - what changes is the stack usage (how deep you are at that point). Note, however, that it's irrelevant since the new code in R-devel doesn't use the stack anymore.
Cheers,
Simon
> For example, if one simply wrap
>
> terms(as.formula("h~ X1*X2*X3*X4*X5*X6*X7*X8*X9*X10*X11*X12*X13*X14*X15*X16")) in test = function() { terms(as.formula("h~ X1*X2*X3*X4*X5*X6*X7*X8*X9*X10*X11*X12*X13*X14*X15*X16"))
> }
>
> It won't cause the problem in the debug mode. Only in the bug report I filed https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=15013 it becomes the problem.
>
> Knowing the limit for different running environment: R vinila, different debug modes, would help to understand it.
>
> Best,
> Peng
>
> On 08/10/2012 05:16 PM, Simon Urbanek wrote:
>> On Aug 10, 2012, at 4:08 PM, Martin Morgan wrote:
>>
>>> On 08/10/2012 12:04 PM, peter dalgaard wrote:
>>>> Not to spoil your fun, but this is getting a bit off-topic for R-help. If you wish to continue the debugging process in public, I think you should move to R-devel.
>>>>
>>>> Also, it sounds like the problem is in the glmulti package, so you might want to involve its maintainer at some point.
>>>
>>>
>>> actually valgrind complains a lot about rJava, though that can be confusing; the example reproduces, so amenable to investigation. glmulti doesn't have C code and the only unusual dependency is rJava
>>>
>>
>> Actually, this crashes is vanilla R (given low enough stack limit, e.g. ulimit -s 1024) - simply try
>>
>> terms(as.formula("h~ X1*X2*X3*X4*X5*X6*X7*X8*X9*X10*X11*X12*X13*X14*X15*X16"))
>>
>> which is what the code below runs. The problem is that it causes a very deep recursion so depending on the stack it will fail (interestingly, it will segfault in the call to R_CheckStack which is paradoxical :)). I think the proper fix would be to use iteration in StripTerm instead of recursion ...
>>
>> Cheers,
>> Simon
>>
>> FWIW Java is in general not easily traceable through valgrind, unfortunately, because it uses a lot of dirty tricks like JIT and other things. There are some hints in the valgrind FAQ 3.4, though.
>>
>>
>>>> sessionInfo()
>>> R version 2.15.1 Patched (2012-06-22 r59603)
>>> Platform: x86_64-unknown-linux-gnu (64-bit)
>>>
>>> locale:
>>> [1] en_US.UTF-8
>>>
>>> attached base packages:
>>> [1] stats graphics grDevices utils datasets methods base
>>>
>>> other attached packages:
>>> [1] glmulti_1.0.4 rJava_0.9-3
>>>
>>>
>>> Here's the stack trace starting a little below the top
>>>
>>> #21190 0x00007ffff78b3b88 in Rf_usemethod (generic=0x9f0c80 "terms", obj=0xbcd040, call=0xbf6fb0, args=0x604d18, rho=0xbd9e70, callrho=0xbf88e8, defrho=0x1a873f0,
>>> ans=0x7fffffffb1a8) at /home/mtmorgan/src/R-2-15-branch/src/main/objects.c:339
>>> 339 *ans = applyMethod(t, sxp, matchedarg, rho, newrho);
>>> (gdb)
>>> #21189 0x00007ffff78b3009 in applyMethod (call=0xbca188, op=0xbf67f0, args=0xbdad30, rho=0xbd9e70, newrho=0xbca118) at /home/mtmorgan/src/R-2-15-branch/src/main/objects.c:125
>>> 125 ans = applyClosure(call, op, args, rho, newrho);
>>> (gdb)
>>> #21188 0x00007ffff784c574 in Rf_applyClosure (call=0xbca188, op=0xbf67f0, arglist=0xbdad30, rho=0xbd9e70, suppliedenv=0xbca118)
>>> at /home/mtmorgan/src/R-2-15-branch/src/main/eval.c:859
>>> 859 PROTECT(tmp = eval(body, newrho));
>>> (gdb)
>>> #21187 0x00007ffff784b4b5 in Rf_eval (e=0xbf52e0, rho=0xbc8f30) at /home/mtmorgan/src/R-2-15-branch/src/main/eval.c:466
>>> 466 tmp = PRIMFUN(op) (e, op, CDR(e), rho);
>>> (gdb)
>>> #21186 0x00007ffff784e12d in do_begin (call=0xbf52e0, op=0x610710, args=0xbee388, rho=0xbc8f30) at /home/mtmorgan/src/R-2-15-branch/src/main/eval.c:1413
>>> 1413 s = eval(CAR(args), rho);
>>> (gdb)
>>> #21185 0x00007ffff784b4b5 in Rf_eval (e=0xbee3c0, rho=0xbc8f30) at /home/mtmorgan/src/R-2-15-branch/src/main/eval.c:466
>>> 466 tmp = PRIMFUN(op) (e, op, CDR(e), rho);
>>> (gdb)
>>> #21184 0x00007ffff784ef32 in do_set (call=0xbee3c0, op=0x610908, args=0xbedbc8, rho=0xbc8f30) at /home/mtmorgan/src/R-2-15-branch/src/main/eval.c:1715
>>> 1715 s = eval(CADR(args), rho);
>>> (gdb)
>>> #21183 0x00007ffff784b4b5 in Rf_eval (e=0xbedc70, rho=0xbc8f30) at /home/mtmorgan/src/R-2-15-branch/src/main/eval.c:466
>>> 466 tmp = PRIMFUN(op) (e, op, CDR(e), rho);
>>> (gdb)
>>> #21182 0x00007ffff78b2979 in do_internal (call=0xbedc70, op=0x611620, args=0xbc9518, env=0xbc8f30) at /home/mtmorgan/src/R-2-15-branch/src/main/names.c:1228
>>> 1228 ans = PRIMFUN(INTERNAL(fun)) (s, INTERNAL(fun), args, env);
>>> (gdb)
>>> #21181 0x00007ffff78acdea in do_termsform (call=0xbedf48, op=0x6329f0, args=0xbc9518, rho=0xbc8f30) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:838
>>> 838 PROTECT(formula = EncodeVars(CAR(args)));
>>> (gdb)
>>> #21180 0x00007ffff78ac4e6 in EncodeVars (formula=0xbcd040) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:629
>>> 629 return EncodeVars(CADDR(formula));
>>> (gdb)
>>> #21179 0x00007ffff78ac5c5 in EncodeVars (formula=0xbcd2a8) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:641
>>> 641 return CrossTerms(CADR(formula), CADDR(formula));
>>> (gdb)
>>> #21178 0x00007ffff78abafb in CrossTerms (left=0xbcd318, right=0xd7e338) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:459
>>> 459 PROTECT(left = EncodeVars(left));
>>> (gdb)
>>> #21177 0x00007ffff78ac5c5 in EncodeVars (formula=0xbcd318) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:641
>>> 641 return CrossTerms(CADR(formula), CADDR(formula));
>>> (gdb)
>>> #21176 0x00007ffff78abc24 in CrossTerms (left=0xbc5558, right=0x16b6e78) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:471
>>> 471 return TrimRepeats(left);
>>> (gdb)
>>> #21175 0x00007ffff78ab934 in TrimRepeats (list=0xbc5558) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:404
>>> 404 SETCDR(list, TrimRepeats(StripTerm(CAR(list), CDR(list))));
>>> (gdb)
>>> #21174 0x00007ffff78ab88d in StripTerm (term=0x121c1d8, list=0xbc5590) at /home/mtmorgan/src/R-2-15-branch/src/main/model.c:385
>>> 385 tail = StripTerm(term, CDR(list));
>>>
>>> and so on down to the bottom
>>>
>>> valgrind says this
>>>
>>> ==14352== Invalid write of size 4
>>> ==14352== at 0xD4BC2BD: ???
>>> ==14352== by 0xD4AC437: ???
>>> ==14352== by 0xBDE21AE: JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBDE1A44: JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBDA5A7E: instanceKlass::call_class_initializer_impl(instanceKlassHandle, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBDA5AF9: instanceKlass::call_class_initializer(Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBDA5CF1: instanceKlass::initialize_impl(instanceKlassHandle, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBDA6168: instanceKlass::initialize(Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBDA5FB2: instanceKlass::initialize_impl(instanceKlassHandle, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBDA6168: instanceKlass::initialize(Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xC0BDB8D: Threads::create_vm(JavaVMInitArgs*, bool*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== by 0xBE00509: JNI_CreateJavaVM (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
>>> ==14352== Address 0x7feff31e0 is not stack'd, malloc'd or (recently) free'd
>>>
>>>
>>>> -pd
>>>>
>>>> On Aug 10, 2012, at 19:51 , Zhang, Peng wrote:
>>>>
>>>>> Thanks! It is interesting that Windows has pointed the problem to Java.
>>>>> So it is probable that how I did my debug led me to the wrong direction.
>>>>>
>>>>> Since I was unsure how to debug S4 class, I copied the source implement
>>>>> R function from glmulti package into my testing program.
>>>>>
>>>>> The segmentation fault will appear when I step to "neotete =
>>>>> terms(as.formula(paste("h~",paste(preds, collapse="*"))))". That is why
>>>>> I have asked my initial question on the board.
>>>>>
>>>>> The more interesting thing is that if I don't run library(glmulti), but
>>>>> just the following lines, I won't have problem when I stepped to that
>>>>> terms function.
>>>>>
>>>>> I am totally lost. It is not clear to me how to debug S4 class.
>>>>>
>>>>> Best,
>>>>> Peng
>>>>>
>>>>> ### beginning
>>>>> library(glmulti)
>>>>>
>>>>> testdata = cbind(Y=rnorm(100), data.frame(matrix(rnorm(100*50), ncol = 50)))
>>>>>
>>>>> glmulti =
>>>>> function(y, xr, data, exclude, name, intercept, marginality , bunch,
>>>>> chunk, chunks,
>>>>> level, minsize, maxsize, minK, maxK,
>>>>> method,crit,confsetsize,popsize,mutrate,
>>>>> sexrate,imm, plotty, report, deltaM, deltaB, conseq,
>>>>> fitfunction, resumefile, includeobjects, ...) {
>>>>> if (missing(data))
>>>>> tete = terms(y)
>>>>> else
>>>>> tete = terms(y, data=data)
>>>>> oo = attr(tete,"order")
>>>>> dep = as.character(attr(tete,"variables"))[2]
>>>>> int = attr(tete,"intercept")
>>>>> preds = as.character(attr(tete,"variables"))[-(1:2)]
>>>>> if (level==2 && max(oo)>1) {
>>>>> # get all possible interactions
>>>>> interac = attr(tete,"term.labels")[oo==2]
>>>>> neotete = terms(as.formula(paste("h~",paste(preds, collapse="*"))))
>>>>> neointerac= attr(neotete,"term.labels")[attr(neotete,"order")==2]
>>>>> # get exclusions
>>>>> for (i in interac)
>>>>> neointerac=neointerac[neointerac!=i]
>>>>> # same for main effects
>>>>> mama = attr(tete,"term.labels")[oo==1]
>>>>> exma = preds
>>>>> for (j in mama)
>>>>> exma = exma[exma!=j]
>>>>> exma = c(exma,neointerac)
>>>>> } else {
>>>>> preds = attr(tete,"term.labels")[oo==1]
>>>>> exma=c(1)
>>>>> }
>>>>> call = match.call()
>>>>> call[[match("y", names(call))]] = dep
>>>>> call[[length(names(call))+1]] = preds
>>>>> names(call)[length(names(call))] ="xr"
>>>>> call[[length(names(call))+1]] = exma
>>>>> names(call)[length(names(call))] ="exclude"
>>>>>
>>>>> if (missing(data)) {
>>>>> call[[length(names(call))+1]] = environment(y)
>>>>> names(call)[length(names(call))] ="data"
>>>>> }
>>>>> eval(call)
>>>>> }
>>>>>
>>>>> debug(glmulti)
>>>>>
>>>>> glmulti(Y~(X1+X2+X3+X4+X5+X6+X7+X8+X9+X10+X11+X12+X13+X14+X15)*X16,
>>>>> xr=NULL, data = testdata, level = 2)
>>>>> ###end
>>>>>
>>>>> On 08/10/2012 11:47 AM, Henric (Nilsson) Winell wrote:
>>>>>> On 2012-08-10 15:42, Zhang, Peng wrote:
>>>>>>
>>>>>>> You are right. I am running Arch Linux. However, I obtained a
>>>>>>> segmentation directly, so didn't know where to find the bug??
>>>>>>>
>>>>>>>> library("glmulti")
>>>>>>> Loading required package: rJava
>>>>>>>> testdata = cbind(Y=rnorm(100), data.frame(matrix(rnorm(100*50), ncol
>>>>>>> = 50)))
>>>>>>>> glmulti(Y~(X1+X2+X3+X4+X5+X6+X7+X8+X9+X10+X11+X12+X13+X14+X15)*X16,
>>>>>>> data= testdata, level = 2)
>>>>>>> Segmentation fault
>>>>>>>
>>>>>>> Is this information "Error in .jnew("glmulti/ModelGenerator", y,
>>>>>>> jarray(xc), .jarray(xq), : java.lang.ArrayIndexOutOfBoundsException:
>>>>>>> 15" only in Windows, or did you see it under Fedora as well?
>>>>>> Windows only. Fedora just returned "Segmentation fault".
>>>>>>
>>>>>>
>>>>>> //Henric
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Thank you!
>>>>>>>
>>>>>>> Peng
>>>>>>>
>>>>>>> BTW:
>>>>>>>
>>>>>>> $ uname -a
>>>>>>> Linux Precision 3.4.7-1-ARCH #1 SMP PREEMPT Sun Jul 29 22:02:56 CEST
>>>>>>> 2012 x86_64 GNU/Linux
>>>>>>>
>>>>>>>> sessionInfo()
>>>>>>> R version 2.15.1 (2012-06-22)
>>>>>>> Platform: x86_64-unknown-linux-gnu (64-bit)
>>>>>>>
>>>>>>> locale:
>>>>>>> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>>>>>>> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>>>>>>> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>>>>>>> [7] LC_PAPER=C LC_NAME=C
>>>>>>> [9] LC_ADDRESS=C LC_TELEPHONE=C
>>>>>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>>>>>>
>>>>>>> attached base packages:
>>>>>>> [1] stats graphics grDevices utils datasets methods base
>>>>>>>
>>>>>>>
>>>>>>> On 08/10/2012 09:25 AM, Henric (Nilsson) Winell wrote:
>>>>>>>> On 2012-08-10 06:10, Zhang, Peng wrote:
>>>>>>>>
>>>>>>>>> Thanks to both for your reply.
>>>>>>>>>
>>>>>>>>> library(glmulti)
>>>>>>>>> testdata = cbind(Y=rnorm(100), data.frame(matrix(rnorm(100*50), ncol
>>>>>>>>> = 50)))
>>>>>>>>> glmulti(Y~(X1+X2+X3+X4+X5+X6+X7+X8+X9+X10+X11+X12+X13+X14+X15)*X16,
>>>>>>>>> data
>>>>>>>>> = testdata, level = 2)
>>>>>>>>>
>>>>>>>>> This is reproducible to get a segmentation fault.
>>>>>>>> There's some information missing here, notably the output from
>>>>>>>> 'sessionInfo()', but I guess this is under GNU/Linux. I can confirm a
>>>>>>>> seg. fault under Fedora FC17 x86_64.
>>>>>>>>
>>>>>>>> Under Windows 7 64-bit, however, we get some further info:
>>>>>>>>
>>>>>>>>> library("glmulti")
>>>>>>>> Loading required package: rJava
>>>>>>>>> testdata = cbind(Y=rnorm(100), data.frame(matrix(rnorm(100*50), ncol
>>>>>>>> = 50)))
>>>>>>>>> glmulti(Y~(X1+X2+X3+X4+X5+X6+X7+X8+X9+X10+X11+X12+X13+X14+X15)*X16,
>>>>>>>> data
>>>>>>>> + = testdata, level = 2)
>>>>>>>> Initialization...
>>>>>>>> Error in .jnew("glmulti/ModelGenerator", y, .jarray(xc),
>>>>>>>> .jarray(xq), :
>>>>>>>> java.lang.ArrayIndexOutOfBoundsException: 15
>>>>>>>>> sessionInfo()
>>>>>>>> R version 2.15.1 Patched (2012-08-06 r60178)
>>>>>>>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>>>>>>>>
>>>>>>>> locale:
>>>>>>>> [1] LC_COLLATE=Swedish_Sweden.1252 LC_CTYPE=Swedish_Sweden.1252
>>>>>>>> [3] LC_MONETARY=Swedish_Sweden.1252 LC_NUMERIC=C
>>>>>>>> [5] LC_TIME=Swedish_Sweden.1252
>>>>>>>>
>>>>>>>> attached base packages:
>>>>>>>> [1] stats graphics grDevices utils datasets methods base
>>>>>>>>
>>>>>>>> other attached packages:
>>>>>>>> [1] glmulti_1.0.4 rJava_0.9-3
>>>>>>>> So, this doesn't seem to be a bug in R and is thus likely to need the
>>>>>>>> attention of the 'glmulti' package's maintainer.
>>>>>>>>
>>>>>>>>
>>>>>>>> HTH,
>>>>>>>> Henric
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> But I have troubles to
>>>>>>>>> extract the exact information from this S4 class to make a simpler
>>>>>>>>> example because of my limited knowledge on S4 class. The author of the
>>>>>>>>> package is busy at the moment, and does not have time to look into it.
>>>>>>>>>
>>>>>>>>> Peng
>>>>>>>>>
>>>>>>>>> On 08/09/2012 10:25 PM, R. Michael Weylandt
>>>>>>>>> <michael.weylandt at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> On Aug 9, 2012, at 9:14 PM, David Winsemius <dwinsemius at comcast.net>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Aug 9, 2012, at 4:56 PM, Zhang, Peng wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>
>>>>>>>>>>>> I had a R segmentation fault, and then invoked debug mode and ran
>>>>>>>>>>>> step
>>>>>>>>>>>> by step.
>>>>>>>>>>>> 2. Why does the same function behave differently under debug and
>>>>>>>>>>>> regular
>>>>>>>>>>>> mode?
>>>>>>>>>>> I cannot help you there.
>>>>>>>>>>>
>>>>>>>>>> Though a reproducible segfault is certainly worth a bug report if
>>>>>>>>>> you can do so, in debug or regular modes.
>>>>>>>>>>
>>>>>>>>>> You may wish to search stackoverflow for tips on how to make a great
>>>>>>>>>> reproducible example in R.
>>>>>>>>>>
>>>>>>>>>> Michael
>>>>>>>>> **********************************************************
>>>>>>>>> Electronic Mail is not secure, may not be read every day, and should
>>>>>>>>> not be used for urgent or sensitive issues
>>>>>>>>>
>>>>>>>>> ______________________________________________
>>>>>>>>> R-help at r-project.org mailing list
>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>>>>>> PLEASE do read the posting guide
>>>>>>>>> http://www.R-project.org/posting-guide.html
>>>>>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>>>>>>
>>>>>>> **********************************************************
>>>>>>> Electronic Mail is not secure, may not be read every day, and should
>>>>>>> not be used for urgent or sensitive issues
>>>>>>>
>>>>>>>
>>>>> **********************************************************
>>>>> Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues
>>>>>
>>>>> ______________________________________________
>>>>> R-help at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>> --
>>> Computational Biology / Fred Hutchinson Cancer Research Center
>>> 1100 Fairview Ave. N.
>>> PO Box 19024 Seattle, WA 98109
>>>
>>> Location: Arnold Building M1 B861
>>> Phone: (206) 667-2793
>>>
>>>
> **********************************************************
> Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues
>
>
More information about the R-devel
mailing list