[R-sig-ME] current r-forge version fails R CMD check ... ?
Ben Bolker
bolker at ufl.edu
Sat Aug 1 21:12:18 CEST 2009
Does this suggest a memory/pointer reference problem (the only way I
can think of getting non-deterministic behavior of this type) ... ? ugh,
ugh, ugh.
Tried running with valgrind, but nothing pops up.
After running the example below to create D, I can get two different
results from the *same* lmer call ...
> table(replicate(40,ranef(lmer(y~(x1+x2)|ff,data=D))$ff[1,1]))
8.35055995996088 8.48042553563304
17 23
I am also worried (without much justification) that the problem might
lie in Matrix, which is even Deeper Magic to me than lme4 ...
Ben
Reinhold Kliegl wrote:
> When I run Martin's example several times, using "set.seed(1)" before
> each run, I get all possible outcomes:
> (a) Error for m0 vs. m1, (b) Error for m2 vs m3, and (c) no error.
>
> Reinhold
>
> other attached packages:
> [1] lme4_0.999375-32 Matrix_0.999375-30 lattice_0.17-25
>
>> ##------------------------------------------
>> # Maechler 01-08-09
>> set.seed(1)
>> ##----------------------------------------------------
>> D <- data.frame(y= rnorm(20,10), ff = gl(4,5),
> + x1=rnorm(20,3), x2=rnorm(20,7),
> + x3=rnorm(20,1))
>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>> m1 <- lmer(y ~ x1 + x2|ff , data = D)
>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>> stopifnot(identical(ranef(m0), ranef(m1)),
> + identical(ranef(m2), ranef(m3)))
> Fehler: identical(ranef(m2), ranef(m3)) is not TRUE
> + cat("Ok\n")
> Ok
>> ##------------------------------------------
>> # Maechler 01-08-09
>> set.seed(1)
>> ##----------------------------------------------------
>> D <- data.frame(y= rnorm(20,10), ff = gl(4,5),
> + x1=rnorm(20,3), x2=rnorm(20,7),
> + x3=rnorm(20,1))
>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>> m1 <- lmer(y ~ x1 + x2|ff , data = D)
>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>> stopifnot(identical(ranef(m0), ranef(m1)),
> + identical(ranef(m2), ranef(m3)))
> Fehler: identical(ranef(m0), ranef(m1)) is not TRUE
> + cat("Ok\n")
> Ok
>> ##------------------------------------------
>> # Maechler 01-08-09
>> set.seed(1)
>> ##----------------------------------------------------
>> D <- data.frame(y= rnorm(20,10), ff = gl(4,5),
> + x1=rnorm(20,3), x2=rnorm(20,7),
> + x3=rnorm(20,1))
>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>> m1 <- lmer(y ~ x1 + x2|ff , data = D)
>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>> stopifnot(identical(ranef(m0), ranef(m1)),
> + identical(ranef(m2), ranef(m3)))
>> cat("Ok\n")
> Ok
>> ##------------------------------------------
>
>
> On Sat, Aug 1, 2009 at 7:53 PM, Ben Bolker<bolker at ufl.edu> wrote:
>> Thanks, Reinhold, I'm glad I'm not completely nuts. With Doug Bates
>> (quite reasonably) occupied with other things, it strikes me it might be
>> a little hard to dig deep enough into the guts to see what's going on
>> ... I will see how far I can get, but this is the kind of problem where
>> **if** we understood what was going on and it looked hard to fix, it
>> would seem reasonable to replace the "must be identical" criterion with
>> "abs(difference)<1e-7" or some such in the tests ...
>>
>> Ben
>>
>> Reinhold Kliegl wrote:
>>> Just updated to Matrix_0.999375-30. The previous problem persists and
>>> now it also reports:
>>> Fehler: identical(ranef(om2), ranef(om3)) is not TRUE
>>>
>>> Reinhold
>>>
>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>> + identical(deviance(om2), deviance(om3)))
>>> Fehler: identical(ranef(om2), ranef(om3)) is not TRUE
>>> + if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>> + stop("offset does not change the fixed effects")
>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>> Time elapsed: 13.588 0.399 14.297 0 0
>>>> sessionInfo()
>>> R version 2.9.1 (2009-06-26)
>>> i386-apple-darwin8.11.1
>>>
>>> locale:
>>> de_DE.UTF-8/en_US.UTF-8/C/C/de_DE.UTF-8/de_DE.UTF-8
>>>
>>> attached base packages:
>>> [1] stats graphics grDevices utils datasets methods
>>> [7] base
>>>
>>> other attached packages:
>>> [1] lme4_0.999375-31 Matrix_0.999375-30 lattice_0.17-25
>>>
>>>
>>> On Sat, Aug 1, 2009 at 7:37 PM, Reinhold
>>> Kliegl<reinhold.kliegl at gmail.com> wrote:
>>>> Ben's problem shows up with my implementation, too. Info below.
>>>>
>>>> Reinhold
>>>>
>>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>> + identical(ranef(m2), ranef(m3)),
>>>> + inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D), error =
>>>> function(e)e),"error"))
>>>> CHOLMOD error: xG˝ LÛR
>>>> Fehler: identical(ranef(m0), ranef(m1)) is not TRUE
>>>> Zusätzlich: Warnmeldung:
>>>> In Ops.factor(ff, x1) : + nicht sinnvoll für Faktoren
>>>> +
>>>>> ## Check the use of offset
>>>>> om2 <- lmer(y ~ x1 + (x2|ff), data = D, offset = x3)
>>>>> om3 <- lmer(y ~ x1 + (x2|ff) + offset(x3), data = D)
>>>>>
>>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>>> + identical(deviance(om2), deviance(om3)))
>>>>> if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>>> + stop("offset does not change the fixed effects")
>>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>>> Time elapsed: 11.608 0.369 12.353 0 0
>>>>> sessionInfo()
>>>> R version 2.9.1 (2009-06-26)
>>>> i386-apple-darwin8.11.1
>>>>
>>>> locale:
>>>> de_DE.UTF-8/en_US.UTF-8/C/C/de_DE.UTF-8/de_DE.UTF-8
>>>>
>>>> attached base packages:
>>>> [1] stats graphics grDevices utils datasets methods
>>>> [7] base
>>>>
>>>> other attached packages:
>>>> [1] lme4_0.999375-31 Matrix_0.999375-29 lattice_0.17-25
>>>>
>>>> loaded via a namespace (and not attached):
>>>> [1] grid_2.9.1
>>>>
>>>> On Sat, Aug 1, 2009 at 6:31 PM, Ben Bolker<bolker at ufl.edu> wrote:
>>>>> I don't mind it being public.
>>>>>
>>>>> I got similar results with the CRAN lme4 (0.999375-31),
>>>>> with Matrix ...-30. BATCH fails on m2 != m3 (consistently);
>>>>> source() fails on m0 != m1.
>>>>>
>>>>> I'm probably doing something really really dumb, would appreciate
>>>>> anyone else who can try this on their systems ...
>>>>>
>>>>> If you don't feel like downloading or running all of lmer-1.R, the
>>>>> following code chunk should demonstrate the problem ...
>>>>>
>>>>> =================
>>>>> library(lme4)
>>>>>
>>>>> set.seed(1)
>>>>> ## Wrong formula gave a seg.fault at times:
>>>>> D <- data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>> x1=rnorm(20,3), x2=rnorm(20,7),
>>>>> x3=rnorm(20,1))
>>>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>> m1 <- lmer(y ~ x1 + x2|ff , data = D)
>>>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>> identical(ranef(m2), ranef(m3)),
>>>>> inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D), error =
>>>>> function(e)e),
>>>>> "error"))
>>>>>
>>>>> ## Check the use of offset
>>>>> om2 <- lmer(y ~ x1 + (x2|ff), data = D, offset = x3)
>>>>> om3 <- lmer(y ~ x1 + (x2|ff) + offset(x3), data = D)
>>>>>
>>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>>>> identical(deviance(om2), deviance(om3)))
>>>>> if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>>>> stop("offset does not change the fixed effects")
>>>>>
>>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>>>>
>>>>>
>>>>> Martin Maechler wrote:
>>>>>> Hi Ben,
>>>>>> as you took this "private", I'd like at least Doug Bates
>>>>>> to be in the CC ..
>>>>>> Personally I would prefer to have this continue in the R-SIG-ME list
>>>>>> rather than privately... I'll be pretty offline from now till Monday
>>>>>> in any case
>>>>>>
>>>>>> On Fri, Jul 31, 2009 at 20:17, Ben Bolker<bolker at ufl.edu> wrote:
>>>>>>> Martin Maechler wrote:
>>>>>>>>>>>>> "BB" == Ben Bolker <bolker at ufl.edu>
>>>>>>>>>>>>> on Thu, 30 Jul 2009 17:30:17 -0400 writes:
>>>>>>>> BB> When I use the latest r-forge version of lme4
>>>>>>>> BB> ( 0.999375-32 ) it seems to fail R CMD check on a tiny
>>>>>>>> BB> numerical mismatch of two objects that are supposed
>>>>>>>> BB> (??) to be identical (I also
>>>>>>>> BB> get a mangled CHOLMOD error message, but I suspect that
>>>>>>>> BB> comes from somewhere within Matrix ...)
>>>>>>>>
>>>>>>>> yes, and those should be gone with the version of Matrix
>>>>>>>> (0.999375-30) of two days ago.
>>>>>>>>
>>>>>>>> BB> can anyone confirm?
>>>>>>>>
>>>>>>>> No. To the contrary.
>>>>>>>> I have had a slightly updated version of tests/lmer-1.Rout.save
>>>>>>>> ready to be committed for a while, but that's only trivial
>>>>>>>> changes.
>>>>>>>>
>>>>>>>> and below, from your sessionInfo(), it looks like you are using
>>>>>>>> a current version of R and packages ...
>>>>>>>> hmm ...
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Martin
>>>>>>>>
>>>>>>>>
>>>>>>>> BB> can anyone confirm? any ideas for a fix?
>>>>>>>>
>>>>>>>>
>>>>>>>> BB> The offending mismatch between ranef(m2) and ranef(m3)
>>>>>>>> BB> is very small ...
>>>>>>>>
>>>>>>>> well; it's interesting that the offending mismatch in the error
>>>>>>>> message below is between m0 and m1, ...
>>>>>>> hmmm indeed. Maybe I was already hacking things. I have
>>>>>>> (1) updated Matrix, (2) installed lme4 directly from r-forge.
>>>>>>> sessionInfo() says
>>>>>>>
>>>>>>> lme4_0.999375-32 Matrix_0.999375-30
>>>>>>>
>>>>>>> in ../tests, I do
>>>>>>>
>>>>>>> R --vanilla
>>>>>>> library(lme4)
>>>>>>> source("lmer-1.R",echo=TRUE)
>>>>>>>
>>>>>>> or
>>>>>>>
>>>>>>> R CMD BATCH --vanilla lmer-1.R
>>>>>>>
>>>>>>> oddly, the second (BATCH) always fails on m0/m1; the
>>>>>>> first (source) fails at different comparisons (sometimes m0/m1;
>>>>>>> sometimes m2/m3; sometimes om2/om3 in the next section ... ???
>>>>>> I just can't understand how that *can* happen.
>>>>>> It would mean that the algorithms used were slightly "random", or
>>>>>> e.g. using slightly different precision depending on memory
>>>>>> allocation, or ??,
>>>>>> ???
>>>>>>
>>>>>> As I said i the first e-mail: The slightly different formula should
>>>>>> produce absolutely identical matrices and vectors which define the
>>>>>> loglikelihood (or RE-LogLik.) and then the minimization really should
>>>>>> be 100% reproducible on a given R+Platform+Installed-Packages setup.
>>>>>>
>>>>>> I assume you have tried the same with the CRAN-version of lme4 ...
>>>>>> which has exactly the same tests/lmer-1.R ?
>>>>>> ....
>>>>>> the phenomenon looks so illogical, I even start to wonder if it's a
>>>>>> bug in your computer (hardware-low-level software combination)?
>>>>>> Maybe you could ask again on R-SIg-ME if others could reproduce?
>>>>>>
>>>>>>>> BTW: Have you noticed that we (Doug Bates and I, when at the
>>>>>>>> useR/DSC meetings) have moved the former 'allcoef' branch into a
>>>>>>>> ``regular R-forge package'' called 'lme4a'
>>>>>>> yes.
>>>>>>>> But yes, that definitely does not pass 'CMD check at the moment'.
>>>>>>>>
>>>>>>>> >> getwd()
>>>>>>>> BB> [1] "/home/ben/lib/R/pkgs/lme4/pkg/lme4/tests"
>>>>>>>>
>>>>>>>> >> source("lmer-1.R",echo=TRUE)
>>>>>>>>
>>>>>>>> BB> ...
>>>>>>>> >> D <- data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>>>>> BB> x1=rnorm(20,3), x2=rnorm(20,7),
>>>>>>>> BB> x3=rnorm(20,1))
>>>>>>>> >> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>>>>> >> m1 <- lmer(y ~ x1 + x2|ff , data = D)
>>>>>>>>
>>>>>>>> We had added these checks exactly *because* we wanted to be sure
>>>>>>>> that a slightly different use of formulas would lead to the
>>>>>>>> identical 'X', 'Z', .... matrices, and L(theta)
>>>>>>>> parametrizations,
>>>>>>>> so I wonder how your version of lme4 could give different
>>>>>>>> results here....
>>>>>>>>
>>>>>>>> >> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>>>>> >> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>>>>> >> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>>>>> BB> + identical(ranef(m2), ranef(m3)),
>>>>>>>> BB> + inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D) ....
>>>>>>>> BB> [TRUNCATED]
>>>>>>>> BB> CHOLMOD error: =*ᶈ1ñ¿@ÀTôoá¶
>>>>>>>> BB> Error: identical(ranef(m0), ranef(m1)) is not TRUE
>>>>>>>> BB> In addition: Warning message:
>>>>>>>> BB> In Ops.factor(ff, x1) : + not meaningful for factors
>>>>>>>>
>>>>>>>> Note that the cholmod error and warning is from the
>>>>>>>> lmer(y ~ x2|ff + x1, data = D)
>>>>>>>> part {which is wrapped in tryCatch(...)}.
>>>>>>>>
>>>>>>>> Also, if I execute
>>>>>>>>
>>>>>>>> ##----------------------------------------------------
>>>>>>>> D <- data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>>>>> x1=rnorm(20,3), x2=rnorm(20,7),
>>>>>>>> x3=rnorm(20,1))
>>>>>>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>>>>> m1 <- lmer(y ~ x1 + x2|ff , data = D)
>>>>>>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>>>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>>>>> identical(ranef(m2), ranef(m3)))
>>>>>>>> cat("Ok\n")
>>>>>>>> ##----------------------------------------------------
>>>>>>>>
>>>>>>>> many times, I never see a problem.
>>>>>>>>
>>>>>>>> Are you sure you are not using your already-hacked version of
>>>>>>>> lme4 ???
>>>>>>>>
>>>>>>>> Martin Maechler, ETH Zurich
>>>>>>>>
>>>>>>> I'm not 100.0000% sure, but I don't see how I could be ...
>>>>>>>
>>>>>>> Ben
>>>>>>>
>>>>>>>
>>>>> --
>>>>> Ben Bolker
>>>>> Associate professor, Biology Dep't, Univ. of Florida
>>>>> bolker at ufl.edu / www.zoology.ufl.edu/bolker
>>>>> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>>
>>
>> --
>> Ben Bolker
>> Associate professor, Biology Dep't, Univ. of Florida
>> bolker at ufl.edu / www.zoology.ufl.edu/bolker
>> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>>
--
Ben Bolker
Associate professor, Biology Dep't, Univ. of Florida
bolker at ufl.edu / www.zoology.ufl.edu/bolker
GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
More information about the R-sig-mixed-models
mailing list