[R-sig-ME] current r-forge version fails R CMD check ... ?

Reinhold Kliegl reinhold.kliegl at gmail.com
Sat Aug 1 20:57:58 CEST 2009


When I run Martin's example several times, using "set.seed(1)" before
each run, I get all possible outcomes:
(a) Error for m0 vs. m1, (b) Error for m2 vs m3, and (c) no error.

Reinhold

other attached packages:
[1] lme4_0.999375-32   Matrix_0.999375-30 lattice_0.17-25

> ##------------------------------------------
> # Maechler 01-08-09
> set.seed(1)
> ##----------------------------------------------------
> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
+                  x1=rnorm(20,3), x2=rnorm(20,7),
+                  x3=rnorm(20,1))
> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
> stopifnot(identical(ranef(m0), ranef(m1)),
+           identical(ranef(m2), ranef(m3)))
Fehler: identical(ranef(m2), ranef(m3)) is not TRUE
+ cat("Ok\n")
Ok
> ##------------------------------------------
> # Maechler 01-08-09
> set.seed(1)
> ##----------------------------------------------------
> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
+                  x1=rnorm(20,3), x2=rnorm(20,7),
+                  x3=rnorm(20,1))
> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
> stopifnot(identical(ranef(m0), ranef(m1)),
+           identical(ranef(m2), ranef(m3)))
Fehler: identical(ranef(m0), ranef(m1)) is not TRUE
+ cat("Ok\n")
Ok
> ##------------------------------------------
> # Maechler 01-08-09
> set.seed(1)
> ##----------------------------------------------------
> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
+                  x1=rnorm(20,3), x2=rnorm(20,7),
+                  x3=rnorm(20,1))
> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
> stopifnot(identical(ranef(m0), ranef(m1)),
+           identical(ranef(m2), ranef(m3)))
> cat("Ok\n")
Ok
> ##------------------------------------------


On Sat, Aug 1, 2009 at 7:53 PM, Ben Bolker<bolker at ufl.edu> wrote:
>  Thanks, Reinhold, I'm glad I'm not completely nuts.  With Doug Bates
> (quite reasonably) occupied with other things, it strikes me it might be
> a little hard to dig deep enough into the guts to see what's going on
> ...  I will see how far I can get, but this is the kind of problem where
> **if** we understood what was going on and it looked hard to fix, it
> would seem reasonable to replace the "must be identical" criterion with
> "abs(difference)<1e-7" or some such in the tests ...
>
>  Ben
>
> Reinhold Kliegl wrote:
>> Just updated to Matrix_0.999375-30. The previous problem persists and
>> now it also reports:
>> Fehler: identical(ranef(om2), ranef(om3)) is not TRUE
>>
>> Reinhold
>>
>>> stopifnot(identical(ranef(om2), ranef(om3)),
>> +          identical(deviance(om2), deviance(om3)))
>> Fehler: identical(ranef(om2), ranef(om3)) is not TRUE
>> + if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>> +    stop("offset does not change the fixed effects")
>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>> Time elapsed:  13.588 0.399 14.297 0 0
>>> sessionInfo()
>> R version 2.9.1 (2009-06-26)
>> i386-apple-darwin8.11.1
>>
>> locale:
>> de_DE.UTF-8/en_US.UTF-8/C/C/de_DE.UTF-8/de_DE.UTF-8
>>
>> attached base packages:
>> [1] stats     graphics  grDevices utils     datasets  methods
>> [7] base
>>
>> other attached packages:
>> [1] lme4_0.999375-31   Matrix_0.999375-30 lattice_0.17-25
>>
>>
>> On Sat, Aug 1, 2009 at 7:37 PM, Reinhold
>> Kliegl<reinhold.kliegl at gmail.com> wrote:
>>> Ben's problem shows up with my implementation, too. Info below.
>>>
>>> Reinhold
>>>
>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>> +          identical(ranef(m2), ranef(m3)),
>>> +          inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D), error =
>>> function(e)e),"error"))
>>> CHOLMOD error: xG˝ LÛR
>>> Fehler: identical(ranef(m0), ranef(m1)) is not TRUE
>>> Zusätzlich: Warnmeldung:
>>> In Ops.factor(ff, x1) : + nicht sinnvoll für Faktoren
>>> +
>>>> ## Check the use of offset
>>>> om2 <- lmer(y ~ x1 + (x2|ff), data = D, offset = x3)
>>>> om3 <- lmer(y ~ x1 + (x2|ff) + offset(x3), data = D)
>>>>
>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>> +          identical(deviance(om2), deviance(om3)))
>>>> if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>> +    stop("offset does not change the fixed effects")
>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>> Time elapsed:  11.608 0.369 12.353 0 0
>>>> sessionInfo()
>>> R version 2.9.1 (2009-06-26)
>>> i386-apple-darwin8.11.1
>>>
>>> locale:
>>> de_DE.UTF-8/en_US.UTF-8/C/C/de_DE.UTF-8/de_DE.UTF-8
>>>
>>> attached base packages:
>>> [1] stats     graphics  grDevices utils     datasets  methods
>>> [7] base
>>>
>>> other attached packages:
>>> [1] lme4_0.999375-31   Matrix_0.999375-29 lattice_0.17-25
>>>
>>> loaded via a namespace (and not attached):
>>> [1] grid_2.9.1
>>>
>>> On Sat, Aug 1, 2009 at 6:31 PM, Ben Bolker<bolker at ufl.edu> wrote:
>>>>  I don't mind it being public.
>>>>
>>>>  I got similar results with the CRAN lme4 (0.999375-31),
>>>> with Matrix ...-30.  BATCH fails on m2 != m3 (consistently);
>>>> source() fails on m0 != m1.
>>>>
>>>>  I'm probably doing something really really dumb, would appreciate
>>>> anyone else who can try this on their systems ...
>>>>
>>>>  If you don't feel like downloading or running all of lmer-1.R, the
>>>> following code chunk should demonstrate the problem ...
>>>>
>>>> =================
>>>> library(lme4)
>>>>
>>>> set.seed(1)
>>>> ## Wrong formula gave a seg.fault at times:
>>>> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>                 x1=rnorm(20,3), x2=rnorm(20,7),
>>>>                 x3=rnorm(20,1))
>>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>          identical(ranef(m2), ranef(m3)),
>>>>          inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D), error =
>>>> function(e)e),
>>>>                   "error"))
>>>>
>>>> ## Check the use of offset
>>>> om2 <- lmer(y ~ x1 + (x2|ff), data = D, offset = x3)
>>>> om3 <- lmer(y ~ x1 + (x2|ff) + offset(x3), data = D)
>>>>
>>>> stopifnot(identical(ranef(om2), ranef(om3)),
>>>>          identical(deviance(om2), deviance(om3)))
>>>> if (identical(TRUE, all.equal(fixef(m2), fixef(om2))))
>>>>    stop("offset does not change the fixed effects")
>>>>
>>>> cat('Time elapsed: ', proc.time(),'\n') # for ``statistical reasons''
>>>>
>>>>
>>>> Martin Maechler wrote:
>>>>> Hi Ben,
>>>>> as you took this "private", I'd like at least Doug Bates
>>>>> to be in the CC ..
>>>>> Personally I would prefer to have this continue in the R-SIG-ME list
>>>>> rather than privately...  I'll be pretty offline from now till Monday
>>>>> in any case
>>>>>
>>>>> On Fri, Jul 31, 2009 at 20:17, Ben Bolker<bolker at ufl.edu> wrote:
>>>>>> Martin Maechler wrote:
>>>>>>>>>>>> "BB" == Ben Bolker <bolker at ufl.edu>
>>>>>>>>>>>>     on Thu, 30 Jul 2009 17:30:17 -0400 writes:
>>>>>>>     BB> When I use the latest r-forge version of lme4
>>>>>>>     BB> (  0.999375-32 ) it seems to fail R CMD check on a tiny
>>>>>>>     BB> numerical mismatch of two objects that are supposed
>>>>>>>     BB> (??) to be identical (I also
>>>>>>>     BB> get a mangled CHOLMOD error message, but I suspect that
>>>>>>>     BB> comes from somewhere within Matrix ...)
>>>>>>>
>>>>>>> yes, and those should be gone with the version of Matrix
>>>>>>> (0.999375-30) of two days ago.
>>>>>>>
>>>>>>>     BB> can anyone confirm?
>>>>>>>
>>>>>>> No.  To the contrary.
>>>>>>> I have had a slightly updated version of tests/lmer-1.Rout.save
>>>>>>> ready to be committed for a while, but that's only trivial
>>>>>>> changes.
>>>>>>>
>>>>>>> and below, from your sessionInfo(), it looks like you are using
>>>>>>> a current version of R and packages ...
>>>>>>> hmm ...
>>>>>>>
>>>>>>> Regards,
>>>>>>> Martin
>>>>>>>
>>>>>>>
>>>>>>>     BB> can anyone confirm? any ideas for a fix?
>>>>>>>
>>>>>>>
>>>>>>>     BB> The offending mismatch between ranef(m2) and ranef(m3)
>>>>>>>     BB> is very small ...
>>>>>>>
>>>>>>> well; it's interesting that the offending mismatch in the error
>>>>>>> message below is between  m0 and m1,  ...
>>>>>>  hmmm indeed.  Maybe I was already hacking things.  I have
>>>>>> (1) updated Matrix, (2) installed lme4 directly from r-forge.
>>>>>> sessionInfo() says
>>>>>>
>>>>>>  lme4_0.999375-32   Matrix_0.999375-30
>>>>>>
>>>>>> in ../tests, I do
>>>>>>
>>>>>> R --vanilla
>>>>>> library(lme4)
>>>>>> source("lmer-1.R",echo=TRUE)
>>>>>>
>>>>>> or
>>>>>>
>>>>>> R CMD BATCH --vanilla lmer-1.R
>>>>>>
>>>>>>  oddly, the second (BATCH) always fails on m0/m1; the
>>>>>> first (source) fails at different comparisons (sometimes m0/m1;
>>>>>> sometimes m2/m3; sometimes om2/om3 in the next section ... ???
>>>>> I just can't understand how that *can* happen.
>>>>> It would mean that the algorithms used were slightly "random",  or
>>>>> e.g. using slightly different precision depending on memory
>>>>> allocation, or ??,
>>>>> ???
>>>>>
>>>>> As I said i the first e-mail: The slightly different formula should
>>>>> produce absolutely identical matrices and vectors which define the
>>>>> loglikelihood (or RE-LogLik.) and then the minimization really should
>>>>> be 100% reproducible on a given R+Platform+Installed-Packages setup.
>>>>>
>>>>> I assume you have tried the same with the CRAN-version of lme4 ...
>>>>> which has exactly the same tests/lmer-1.R  ?
>>>>> ....
>>>>> the phenomenon looks so illogical,  I even start to wonder if it's a
>>>>> bug in your computer (hardware-low-level software combination)?
>>>>> Maybe you could ask again on R-SIg-ME if others could reproduce?
>>>>>
>>>>>>> BTW: Have you noticed that we (Doug Bates and I, when at the
>>>>>>> useR/DSC meetings) have moved the former 'allcoef' branch into a
>>>>>>> ``regular R-forge package''  called  'lme4a'
>>>>>>  yes.
>>>>>>> But yes, that definitely does not pass 'CMD check at the moment'.
>>>>>>>
>>>>>>>     >> getwd()
>>>>>>>     BB> [1] "/home/ben/lib/R/pkgs/lme4/pkg/lme4/tests"
>>>>>>>
>>>>>>>     >> source("lmer-1.R",echo=TRUE)
>>>>>>>
>>>>>>>     BB> ...
>>>>>>>     >> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>>>>     BB> x1=rnorm(20,3), x2=rnorm(20,7),
>>>>>>>     BB> x3=rnorm(20,1))
>>>>>>>     >> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>>>>     >> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>>>>>>
>>>>>>> We had added these checks exactly *because* we wanted to be sure
>>>>>>> that a slightly different use of formulas would lead to the
>>>>>>> identical 'X', 'Z', .... matrices, and L(theta)
>>>>>>> parametrizations,
>>>>>>> so I wonder how your version of lme4 could give different
>>>>>>> results here....
>>>>>>>
>>>>>>>     >> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>>>>     >> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>>>>     >> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>>>>     BB> +           identical(ranef(m2), ranef(m3)),
>>>>>>>     BB> +           inherits(tryCatch(lmer(y ~ x2|ff + x1, data = D) ....
>>>>>>>     BB> [TRUNCATED]
>>>>>>>     BB> CHOLMOD error: =*ᶈ1ñ¿@ÀTôoá¶
>>>>>>>     BB> Error: identical(ranef(m0), ranef(m1)) is not TRUE
>>>>>>>     BB> In addition: Warning message:
>>>>>>>     BB> In Ops.factor(ff, x1) : + not meaningful for factors
>>>>>>>
>>>>>>> Note that the cholmod error and warning is from the
>>>>>>>    lmer(y ~ x2|ff + x1, data = D)
>>>>>>> part {which is wrapped in  tryCatch(...)}.
>>>>>>>
>>>>>>> Also, if I execute
>>>>>>>
>>>>>>> ##----------------------------------------------------
>>>>>>> D <-  data.frame(y= rnorm(20,10), ff = gl(4,5),
>>>>>>>                  x1=rnorm(20,3), x2=rnorm(20,7),
>>>>>>>                  x3=rnorm(20,1))
>>>>>>> m0 <- lmer(y ~ (x1 + x2)|ff, data = D)
>>>>>>> m1 <- lmer(y ~ x1 + x2|ff  , data = D)
>>>>>>> m2 <- lmer(y ~ x1 + (x2|ff), data = D)
>>>>>>> m3 <- lmer(y ~ (x2|ff) + x1, data = D)
>>>>>>> stopifnot(identical(ranef(m0), ranef(m1)),
>>>>>>>           identical(ranef(m2), ranef(m3)))
>>>>>>> cat("Ok\n")
>>>>>>> ##----------------------------------------------------
>>>>>>>
>>>>>>> many times, I never see a problem.
>>>>>>>
>>>>>>> Are you sure you are not using your already-hacked version of
>>>>>>> lme4 ???
>>>>>>>
>>>>>>> Martin Maechler, ETH Zurich
>>>>>>>
>>>>>>  I'm not 100.0000% sure, but I don't see how I could be ...
>>>>>>
>>>>>>  Ben
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> Ben Bolker
>>>> Associate professor, Biology Dep't, Univ. of Florida
>>>> bolker at ufl.edu / www.zoology.ufl.edu/bolker
>>>> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>
>
> --
> Ben Bolker
> Associate professor, Biology Dep't, Univ. of Florida
> bolker at ufl.edu / www.zoology.ufl.edu/bolker
> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>




More information about the R-sig-mixed-models mailing list