[R-sig-ME] current r-forge version fails R CMD check ... ?

Reinhold Kliegl reinhold.kliegl at gmail.com
Mon Aug 3 15:44:02 CEST 2009


Make this: MacBook Pro and MacBook Air
Reinhold


On Mon, Aug 3, 2009 at 2:58 PM, Bolker,Benjamin Michael<bolker at ufl.edu> wrote:
>
>   Another request to others out there to try to replicate -- perhaps if
> we can see where it works and where it doesn't that will help narrow down
> the problem (and understand why Martin can't replicate)?
>
>  So far we have it with 2.9.1, Matrix_0.999375-30, and lme4
> lme4_0.999375-{30,31,32} (I think), on a MacBook and on Ubuntu 9.04?
>
>  Ben
>
> ________________________________________
> From: Martin Maechler [maechler at stat.math.ethz.ch]
> Sent: Monday, August 03, 2009 6:06 AM
> To: Bolker,Benjamin Michael
> Cc: Reinhold Kliegl; R Mixed Models; Martin Maechler
> Subject: Re: [R-sig-ME] current r-forge version fails R CMD check ... ?
>
>    BB>   Does this suggest a memory/pointer reference problem
>    BB> (the only way I can think of getting non-deterministic
>    BB> behavior of this type) ... ? ugh, ugh, ugh.
>
>
>    BB>   Tried running with valgrind, but nothing pops up.
>
> You could try
>    gctorture()
> as well.  But beware, that needs a lot of patience.
>
>    BB>   After running the example below to create D, I can get
>    BB> two different results from the *same* lmer call ...
>
>    >> table(replicate(40,ranef(lmer(y~(x1+x2)|ff,data=D))$ff[1,1]))
>
>    BB> 8.35055995996088 8.48042553563304 17 23
>
> aha.  This at least seems "logical" to me, given the  identical()
> problems mentioned earlier.
> Since, I've said all the time,  the two ways really must give
> identical results, and using all.equal() instead is really the
> wrong test for the purpose ...
>
>
>    BB>   I am also worried (without much justification) that
>    BB> the problem might lie in Matrix, which is even Deeper
>    BB> Magic to me than lme4 ...
>
> :-)
>
> If I only found a way to reproduce your findings on one of my
> Linux platforms,...
> then I could start digging...
>
> Martin
>
>
>
>    BB> Reinhold Kliegl wrote:
>    >> When I run Martin's example several times, using
>    >> "set.seed(1)" before each run, I get all possible
>    >> outcomes: (a) Error for m0 vs. m1, (b) Error for m2 vs
>    >> m3, and (c) no error.
>    >>
>    >> Reinhold
>    >>
>    >> other attached packages: [1] lme4_0.999375-32
>    >> Matrix_0.999375-30 lattice_0.17-25
>    >>
>    >>> ##------------------------------------------
>>> # Maechler 01-08-09
>>> set.seed(1)
>>> ##----------------------------------------------------
>>> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>> +                  x1=rnorm(20,3), x2=rnorm(20,7),
>> +                  x3=rnorm(20,1))
>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>> stopifnot(identical(ranef(m0), ranef(m1)),
>> +           identical(ranef(m2), ranef(m3)))
>> Fehler: identical(ranef(m2), ranef(m3)) is not TRUE
>> + cat("Ok\n")
>> Ok
>>> ##------------------------------------------
>>> # Maechler 01-08-09
>>> set.seed(1)
>>> ##----------------------------------------------------
>>> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>> +                  x1=rnorm(20,3), x2=rnorm(20,7),
>> +                  x3=rnorm(20,1))
>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>> stopifnot(identical(ranef(m0), ranef(m1)),
>> +           identical(ranef(m2), ranef(m3)))
>> Fehler: identical(ranef(m0), ranef(m1)) is not TRUE
>> + cat("Ok\n")
>> Ok
>>> ##------------------------------------------
>>> # Maechler 01-08-09
>>> set.seed(1)
>>> ##----------------------------------------------------
>>> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>> +                  x1=rnorm(20,3), x2=rnorm(20,7),
>> +                  x3=rnorm(20,1))
>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>> stopifnot(identical(ranef(m0), ranef(m1)),
>> +           identical(ranef(m2), ranef(m3)))
>>> cat("Ok\n")
>> Ok
>>> ##------------------------------------------
>>
>>
>> On Sat, Aug 1, 2009 at 7:53 PM, Ben Bolker<bolker at ufl.edu> wrote:
>>>  Thanks, Reinhold, I'm glad I'm not completely nuts.  With Doug Bates
>>> (quite reasonably) occupied with other things, it strikes me it might be
>>> a little hard to dig deep enough into the guts to see what's going on
>>> ...  I will see how far I can get, but this is the kind of problem where
>>> **if** we understood what was going on and it looked hard to fix, it
>>> would seem reasonable to replace the "must be identical" criterion with
>>> "abs(difference)<1e-7" or some such in the tests ...
>>>
>>>  Ben
>>>
>>> Reinhold Kliegl wrote:
>>>> Just updated to Matrix_0.999375-30. The previous problem persists and
>>>> now it also reports:
>>>> Fehler: identical(ranef(om2), ranef(om3)) is not TRUE
>>>>
>>>> Reinhold
>>>>
>>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>>> +          identical(deviance(om2), deviance(om3)))
>>>> Fehler: identical(ranef(om2), ranef(om3)) is not TRUE
>>>> + if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>>> +    stop("offset does not change the fixed effects")
>>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>>> Time elapsed:  13.588 0.399 14.297 0 0
>>>>> sessionInfo()
>>>> R version 2.9.1 (2009-06-26)
>>>> i386-apple-darwin8.11.1
>>>>
>>>> locale:
>>>> de_DE.UTF-8/en_US.UTF-8/C/C/de_DE.UTF-8/de_DE.UTF-8
>>>>
>>>> attached base packages:
>>>> [1] stats     graphics  grDevices utils     datasets  methods
>>>> [7] base
>>>>
>>>> other attached packages:
>>>> [1] lme4_0.999375-31   Matrix_0.999375-30 lattice_0.17-25
>>>>
>>>>
>>>> On Sat, Aug 1, 2009 at 7:37 PM, Reinhold
>>>> Kliegl<reinhold.kliegl at gmail.com> wrote:
>>>>> Ben's problem shows up with my implementation, too. Info below.
>>>>>
>>>>> Reinhold
>>>>>
>>>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>> +          identical(ranef(m2), ranef(m3)),
>>>>> +          inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D), error =
>>>>> function(e)e),"error"))
>>>>> CHOLMOD error: xG˝ LÛR
>>>>> Fehler: identical(ranef(m0), ranef(m1)) is not TRUE
>>>>> Zusätzlich: Warnmeldung:
>>>>> In Ops.factor(ff, x1) : + nicht sinnvoll für Faktoren
>>>>> +
>>>>>> ## Check the use of offset
>>>>>> om2 <- lmer(y ~ x1 + (x2|ff), data = D, offset = x3)
>>>>>> om3 <- lmer(y ~ x1 + (x2|ff) + offset(x3), data = D)
>>>>>>
>>>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>>>> +          identical(deviance(om2), deviance(om3)))
>>>>>> if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>>>> +    stop("offset does not change the fixed effects")
>>>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>>>> Time elapsed:  11.608 0.369 12.353 0 0
>>>>>> sessionInfo()
>>>>> R version 2.9.1 (2009-06-26)
>>>>> i386-apple-darwin8.11.1
>>>>>
>>>>> locale:
>>>>> de_DE.UTF-8/en_US.UTF-8/C/C/de_DE.UTF-8/de_DE.UTF-8
>>>>>
>>>>> attached base packages:
>>>>> [1] stats     graphics  grDevices utils     datasets  methods
>>>>> [7] base
>>>>>
>>>>> other attached packages:
>>>>> [1] lme4_0.999375-31   Matrix_0.999375-29 lattice_0.17-25
>>>>>
>>>>> loaded via a namespace (and not attached):
>>>>> [1] grid_2.9.1
>>>>>
>>>>> On Sat, Aug 1, 2009 at 6:31 PM, Ben Bolker<bolker at ufl.edu> wrote:
>>>>>>  I don't mind it being public.
>>>>>>
>>>>>>  I got similar results with the CRAN lme4 (0.999375-31),
>>>>>> with Matrix ...-30.  BATCH fails on m2 != m3 (consistently);
>>>>>> source() fails on m0 != m1.
>>>>>>
>>>>>>  I'm probably doing something really really dumb, would appreciate
>>>>>> anyone else who can try this on their systems ...
>>>>>>
>>>>>>  If you don't feel like downloading or running all of lmer-1.R, the
>>>>>> following code chunk should demonstrate the problem ...
>>>>>>
>>>>>> =================
>>>>>> library(lme4)
>>>>>>
>>>>>> set.seed(1)
>>>>>> ## Wrong formula gave a seg.fault at times:
>>>>>> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>>>                 x1=rnorm(20,3), x2=rnorm(20,7),
>>>>>>                 x3=rnorm(20,1))
>>>>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>>> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>>>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>>>          identical(ranef(m2), ranef(m3)),
>>>>>>          inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D), error =
>>>>>> function(e)e),
>>>>>>                   "error"))
>>>>>>
>>>>>> ## Check the use of offset
>>>>>> om2 <- lmer(y ~ x1 + (x2|ff), data = D, offset = x3)
>>>>>> om3 <- lmer(y ~ x1 + (x2|ff) + offset(x3), data = D)
>>>>>>
>>>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>>>>>          identical(deviance(om2), deviance(om3)))
>>>>>> if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>>>>>    stop("offset does not change the fixed effects")
>>>>>>
>>>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>>>>>
>>>>>>
>>>>>> Martin Maechler wrote:
>>>>>>> Hi Ben,
>>>>>>> as you took this "private", I'd like at least Doug Bates
>>>>>>> to be in the CC ..
>>>>>>> Personally I would prefer to have this continue in the R-SIG-ME list
>>>>>>> rather than privately...  I'll be pretty offline from now till Monday
>>>>>>> in any case
>>>>>>>
>>>>>>> On Fri, Jul 31, 2009 at 20:17, Ben Bolker<bolker at ufl.edu> wrote:
>>>>>>>> Martin Maechler wrote:
>>>>>>>>>>>>>> "BB" == Ben Bolker <bolker at ufl.edu>
>>>>>>>>>>>>>>     on Thu, 30 Jul 2009 17:30:17 -0400 writes:
>>>>>>>>>     BB> When I use the latest r-forge version of lme4
>>>>>>>>>     BB> (  0.999375-32 ) it seems to fail R CMD check on a tiny
>>>>>>>>>     BB> numerical mismatch of two objects that are supposed
>>>>>>>>>     BB> (??) to be identical (I also
>>>>>>>>>     BB> get a mangled CHOLMOD error message, but I suspect that
>>>>>>>>>     BB> comes from somewhere within Matrix ...)
>>>>>>>>>
>>>>>>>>> yes, and those should be gone with the version of Matrix
>>>>>>>>> (0.999375-30) of two days ago.
>>>>>>>>>
>>>>>>>>>     BB> can anyone confirm?
>>>>>>>>>
>>>>>>>>> No.  To the contrary.
>>>>>>>>> I have had a slightly updated version of tests/lmer-1.Rout.save
>>>>>>>>> ready to be committed for a while, but that's only trivial
>>>>>>>>> changes.
>>>>>>>>>
>>>>>>>>> and below, from your sessionInfo(), it looks like you are using
>>>>>>>>> a current version of R and packages ...
>>>>>>>>> hmm ...
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Martin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>     BB> can anyone confirm? any ideas for a fix?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>     BB> The offending mismatch between ranef(m2) and ranef(m3)
>>>>>>>>>     BB> is very small ...
>>>>>>>>>
>>>>>>>>> well; it's interesting that the offending mismatch in the error
>>>>>>>>> message below is between  m0 and m1,  ...
>>>>>>>>  hmmm indeed.  Maybe I was already hacking things.  I have
>>>>>>>> (1) updated Matrix, (2) installed lme4 directly from r-forge.
>>>>>>>> sessionInfo() says
>>>>>>>>
>>>>>>>>  lme4_0.999375-32   Matrix_0.999375-30
>>>>>>>>
>>>>>>>> in ../tests, I do
>>>>>>>>
>>>>>>>> R --vanilla
>>>>>>>> library(lme4)
>>>>>>>> source("lmer-1.R",echo=TRUE)
>>>>>>>>
>>>>>>>> or
>>>>>>>>
>>>>>>>> R CMD BATCH --vanilla lmer-1.R
>>>>>>>>
>>>>>>>>  oddly, the second (BATCH) always fails on m0/m1; the
>>>>>>>> first (source) fails at different comparisons (sometimes m0/m1;
>>>>>>>> sometimes m2/m3; sometimes om2/om3 in the next section ... ???
>>>>>>> I just can't understand how that *can* happen.
>>>>>>> It would mean that the algorithms used were slightly "random",  or
>>>>>>> e.g. using slightly different precision depending on memory
>>>>>>> allocation, or ??,
>>>>>>> ???
>>>>>>>
>>>>>>> As I said i the first e-mail: The slightly different formula should
>>>>>>> produce absolutely identical matrices and vectors which define the
>>>>>>> loglikelihood (or RE-LogLik.) and then the minimization really should
>>>>>>> be 100% reproducible on a given R+Platform+Installed-Packages setup.
>>>>>>>
>>>>>>> I assume you have tried the same with the CRAN-version of lme4 ...
>>>>>>> which has exactly the same tests/lmer-1.R  ?
>>>>>>> ....
>>>>>>> the phenomenon looks so illogical,  I even start to wonder if it's a
>>>>>>> bug in your computer (hardware-low-level software combination)?
>>>>>>> Maybe you could ask again on R-SIg-ME if others could reproduce?
>>>>>>>
>>>>>>>>> BTW: Have you noticed that we (Doug Bates and I, when at the
>>>>>>>>> useR/DSC meetings) have moved the former 'allcoef' branch into a
>>>>>>>>> ``regular R-forge package''  called  'lme4a'
>>>>>>>>  yes.
>>>>>>>>> But yes, that definitely does not pass 'CMD check at the moment'.
>>>>>>>>>
>>>>>>>>>     >> getwd()
>>>>>>>>>     BB> [1] "/home/ben/lib/R/pkgs/lme4/pkg/lme4/tests"
>>>>>>>>>
>>>>>>>>>     >> source("lmer-1.R",echo=TRUE)
>>>>>>>>>
>>>>>>>>>     BB> ...
>>>>>>>>>     >> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>>>>>>     BB> x1=rnorm(20,3), x2=rnorm(20,7),
>>>>>>>>>     BB> x3=rnorm(20,1))
>>>>>>>>>     >> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>>>>>>     >> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>>>>>>>>
>>>>>>>>> We had added these checks exactly *because* we wanted to be sure
>>>>>>>>> that a slightly different use of formulas would lead to the
>>>>>>>>> identical 'X', 'Z', .... matrices, and L(theta)
>>>>>>>>> parametrizations,
>>>>>>>>> so I wonder how your version of lme4 could give different
>>>>>>>>> results here....
>>>>>>>>>
>>>>>>>>>     >> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>>>>>>     >> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>>>>>>     >> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>>>>>>     BB> +           identical(ranef(m2), ranef(m3)),
>>>>>>>>>     BB> +           inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D) ....
>>>>>>>>>     BB> [TRUNCATED]
>>>>>>>>>     BB> CHOLMOD error: =*ᶈ1ñ¿@ÀTôoá¶
>>>>>>>>>     BB> Error: identical(ranef(m0), ranef(m1)) is not TRUE
>>>>>>>>>     BB> In addition: Warning message:
>>>>>>>>>     BB> In Ops.factor(ff, x1) : + not meaningful for factors
>>>>>>>>>
>>>>>>>>> Note that the cholmod error and warning is from the
>>>>>>>>>    lmer(y ~ x2|ff + x1, data = D)
>>>>>>>>> part {which is wrapped in  tryCatch(...)}.
>>>>>>>>>
>>>>>>>>> Also, if I execute
>>>>>>>>>
>>>>>>>>> ##----------------------------------------------------
>>>>>>>>> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>>>>>>                  x1=rnorm(20,3), x2=rnorm(20,7),
>>>>>>>>>                  x3=rnorm(20,1))
>>>>>>>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>>>>>> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>>>>>>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>>>>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>>>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>>>>>>           identical(ranef(m2), ranef(m3)))
>>>>>>>>> cat("Ok\n")
>>>>>>>>> ##----------------------------------------------------
>>>>>>>>>
>>>>>>>>> many times, I never see a problem.
>>>>>>>>>
>>>>>>>>> Are you sure you are not using your already-hacked version of
>>>>>>>>> lme4 ???
>>>>>>>>>
>>>>>>>>> Martin Maechler, ETH Zurich
>>>>>>>>>
>>>>>>>>  I'm not 100.0000% sure, but I don't see how I could be ...
>>>>>>>>
>>>>>>>>  Ben
>>>>>>>>
>>>>>>>>
>>>>>> --
>>>>>> Ben Bolker
>>>>>> Associate professor, Biology Dep't, Univ. of Florida
>>>>>> bolker at ufl.edu / www.zoology.ufl.edu/bolker
>>>>>> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>>>
>>>
>>> --
>>> Ben Bolker
>>> Associate professor, Biology Dep't, Univ. of Florida
>>> bolker at ufl.edu / www.zoology.ufl.edu/bolker
>>> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>>>
>
>
> --
> Ben Bolker
> Associate professor, Biology Dep't, Univ. of Florida
> bolker at ufl.edu / www.zoology.ufl.edu/bolker
> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>




More information about the R-sig-mixed-models mailing list