[R-sig-ME] bug in identical()? [Was: Failure to load lme4 on Mac]
Reinhold Kliegl
reinhold.kliegl at gmail.com
Sun Jul 18 12:28:55 CEST 2010
First of all, I really appreciate these efforts.
Changing to R BLAS did not do the trick for me on Mac OS X (10.5.8).
Here is the protocol.
###
Last login: Sun Jul 18 12:04:10 on ttys000
Reinhold:~ kliegl$ bash
~ > cd /Library/Frameworks/R.framework/Resources/lib
/Library/Frameworks/R.framework/Resources/lib > ls
i386 libRblas.vecLib.dylib libreadline.5.2.dylib
libR.dylib libRlapack.dylib libreadline.dylib
libRblas.0.dylib libgcc_s.1.dylib ppc
libRblas.dylib libgfortran.2.dylib x86_64
/Library/Frameworks/R.framework/Resources/lib > ln -sf
libRblas.0.dylib libRblas.dylib
/Library/Frameworks/R.framework/Resources/lib > R
R version 2.11.1 Patched (2010-07-16 r52550)
Copyright (C) 2010 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
< clipped>
> library(lme4)
<clipped>
> y <- (1:6)*pi # 3.14
> x <- (1:6)^2
> group <- gl(2,3)
> for (i in 1:10) {
+ M1 <- lmer (y ~ x + (x | group))
+ M2 <- lmer (y ~ x + (x | group))
+ print(identical(ranef(M1),ranef(M2)))
+ print(identical(coef(M1),coef(M2)))
+ }
[1] TRUE
[1] TRUE
[1] TRUE
[1] TRUE
[1] FALSE
[1] FALSE
[1] TRUE
[1] TRUE
[1] FALSE
[1] FALSE
[1] FALSE
[1] FALSE
[1] TRUE
[1] TRUE
[1] FALSE
[1] FALSE
[1] TRUE
[1] TRUE
[1] TRUE
[1] TRUE
> sessionInfo()
R version 2.11.1 Patched (2010-07-16 r52550)
i386-apple-darwin9.8.0
locale:
[1] de_DE/de_DE/C/C/de_DE/de_DE
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] lme4_0.999375-34 Matrix_0.999375-41 lattice_0.18-8
loaded via a namespace (and not attached):
[1] grid_2.11.1 nlme_3.1-96 stats4_2.11.1
>
#####
Did I do something wrong in getting the correct library? Is there a
command to check whether R BLAS is used?
Using require(lme4) instead of library(lme4) did not make a difference either.
Many thanks again,
Reinhold Kliegl
On Sun, Jul 18, 2010 at 11:35 AM, Daniel Myall <daniel.lists at zeno.co.nz> wrote:
> Hi Simon,
>
> The Apple vecLib BLAS appears to be the cause of the strangeness (at
> least on my machine).
>
> Following:
> http://cran.r-project.org/bin/macosx/RMacOSX-FAQ.html#Which-BLAS-is-used-and-how-can-it-be-changed_003f
>
>
> I changed to the R BLAS and everything now works as it should (i.e., for
> certain source data, lme4 now gives the same solution on multiple runs
> instead of randomly giving one of two different "solutions" on each
> run). The FAQ does note in relation to vecLib BLAS " Although fast, it
> is not under our control and may possibly deliver inaccurate results."
> which appears to be true, at least with lme4.
>
> What BLAS does the CRAN build machine use? If it does use the vecLib
> BLAS is there a case for changing to R BLAS (maybe only when
> building/checking lme4)?
>
> Cheers,
> Daniel
>
> On 18/07/10 9:50 AM, Daniel Myall wrote:
>> Hi Simon,
>>
>> Unfortunately I don't think is the full story (there are actual small
>> differences in the fits).
>>
>> To match the lme4 tests the example should actually be:
>>
>> library(lme4)
>> y<- (1:20)*pi
>> x<- (1:20)^2
>> group<- gl(2,10)
>> for (i in 1:10) {
>> M1<- lmer (y ~ x + ( x | group))
>> M2<- lmer (y ~ x + ( x | group))
>> print(identical(ranef(M1),ranef(M2)))
>> print(ranef(M1)$group-ranef(M2)$group)
>> }
>>
>> Which gives me (the number of TRUE/FALSE and the order change on each
>> run, even if restarting R):
>>
>> [1] FALSE
>> (Intercept) x
>> 1 6.613450e-06 -6.898626e-08
>> 2 -6.613462e-06 6.898637e-08
>> [1] TRUE
>> (Intercept) x
>> 1 0 0
>> 2 0 0
>> [1] FALSE
>> (Intercept) x
>> 1 -6.613450e-06 6.898626e-08
>> 2 6.613462e-06 -6.898637e-08
>> [1] FALSE
>> (Intercept) x
>> 1 6.613450e-06 -6.898626e-08
>> 2 -6.613462e-06 6.898637e-08
>> [1] TRUE
>> (Intercept) x
>> 1 0 0
>> 2 0 0
>> [1] FALSE
>> (Intercept) x
>> 1 -6.613450e-06 6.898626e-08
>> 2 6.613462e-06 -6.898637e-08
>> [1] FALSE
>> (Intercept) x
>> 1 -6.613450e-06 6.898626e-08
>> 2 6.613462e-06 -6.898637e-08
>> [1] FALSE
>> (Intercept) x
>> 1 -6.613450e-06 6.898626e-08
>> 2 6.613462e-06 -6.898637e-08
>> [1] TRUE
>> (Intercept) x
>> 1 0 0
>> 2 0 0
>> [1] TRUE
>> (Intercept) x
>> 1 0 0
>> 2 0 0
>>
>>
>> Although only small differences, I assume lme4 should be deterministic?
>>
>> Cheers,
>> Daniel
>>
>>
>>
>>
>> On 18/07/10 8:23 AM, Simon Urbanek wrote:
>>> Ok, I think I found the issue. I'm not sure why this varies by
>>> platform but the mismatch is due to the @env slot. Two environments
>>> are only identical if it is *the* same environment (i.e. the same
>>> pointer). However, M1 and M2 have different environments. The content
>>> of those environments is identical, but that is irrelevant as it's
>>> not the same pointer. Hence identical(M1, M2) fails (and serialize
>>> comparison succeeds as it cares only about the content).
>>>
>>> So the short story is don't use identical() to compare the models
>>> (unless you remove @env first). The long story raises the question
>>> whether identical() should really return FALSE for environments like
>>>> identical(new.env(),new.env())
>>> [1] FALSE
>>> I can see arguments both ways but for the purposes of comparing
>>> values there should be an option that the above is TRUE.
>>>
>>> To be honest I don't see why this has not shown up on other platforms
>>> as that is a global issue... (I hope this is the full story - I
>>> didn't try all the combinations to see if setting @env to the same
>>> environment will appease identical() for all the models)
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>> On Jul 17, 2010, at 3:49 PM, Simon Urbanek wrote:
>>>
>>>> Daniel,
>>>>
>>>> thanks for the test case. I did run it in valgrind but nothing
>>>> showed up, however ...
>>>>
>>>> I'm starting to have a suspicion that this has something to do with
>>>> identical() - look at this:
>>>>
>>>>> identical(M1,M2)
>>>> [1] FALSE
>>>>> all(serialize(M1,NULL)==serialize(M2,NULL))
>>>> [1] TRUE
>>>>> identical(unserialize(serialize(M1,NULL)),unserialize(serialize(M2,NULL)))
>>>>>
>>>> [1] FALSE
>>>>> identical(unserialize(serialize(M1,NULL)),unserialize(serialize(M1,NULL)))
>>>>>
>>>> [1] FALSE
>>>>
>>>> So I think this may be a bug in identical() mainly because of the
>>>> last one. I'll need to take identical() apart to see where it fails
>>>> ... I'm CCing this to R-devel as the current issue seems more like
>>>> an R issue so more eyes can have a look ...
>>>>
>>>> Cheers,
>>>> Simon
>>>>
>>>>
>>>> [FWIW this is tested in today's R-devel (with valgrind level 2) on
>>>> x86_64 OS X 10.6.4 with lme4 from CRAN and Matrix form R-devel
>>>> Recommended]
>>>>
>>>>
>>>> On Jul 17, 2010, at 4:50 AM, Daniel Myall wrote:
>>>>
>>>>> I've done some further testing (R 2.11.1) and the issue is not
>>>>> limited to Leopard.
>>>>>
>>>>> Using the test:
>>>>>
>>>>> library(lme4)
>>>>> y<- (1:20)*pi
>>>>> x<- (1:20)^2
>>>>> group<- gl(2,10)
>>>>> for (i in 1:10) {
>>>>> M1<- lmer (y ~ x + ( x | group))
>>>>> M2<- lmer (y ~ x + ( x | group))
>>>>> print(identical(M1,M2))
>>>>> }
>>>>>
>>>>> For CRAN lme4 and Matrix:
>>>>>
>>>>> 32 bit on Leopard: R CMD check fails; different results (on most runs)
>>>>> 32 bit on Snow Leopard: R CMD check passes; different results (on
>>>>> some runs).
>>>>> 64 bit on Snow Leopard: R CMD check passes; identical results
>>>>>
>>>>> For SVN version of Matrix with CRAN lme4:
>>>>>
>>>>> 32 bit on Snow Leopard: different results (on all runs).
>>>>> 64 bit on Snow Leopard: different results (on all runs)
>>>>>
>>>>> For SVN version of Matrix with SVN lme4a:
>>>>>
>>>>> 32 bit on Snow Leopard: different results (on all runs).
>>>>> 64 bit on Snow Leopard: identical results
>>>>>
>>>>> I couldn't reproduce on Linux 32/64bit. Is it time to jump into
>>>>> valgrind to try and find the cause?
>>>>>
>>>>> Cheers,
>>>>> Daniel
>>>>>
>>>>>
>>>>>
>>>>> On 17/07/10 5:51 PM, John Maindonald wrote:
>>>>>> In principle, maybe a Snow Leopard version might be posted
>>>>>> as an alternative, if someone can provide one. But I take it
>>>>>> that the issue is now a bit wider than tests that fail on Leopard
>>>>>> vs passing on Snow Leopard?
>>>>>>
>>>>>
>>>
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
More information about the R-sig-mixed-models
mailing list