[R-sig-ME] bug in identical()? [Was: Failure to load lme4 on Mac]
Reinhold Kliegl
reinhold.kliegl at gmail.com
Sun Jul 18 13:20:23 CEST 2010
Actually, what makes a difference on my machine for this test example
is switching to
`R --arch x86_64'
After this switch linking either R BLAS and vecLIB BLAS checked out
fine--at least for 100 tests of the example.
( I think ...).
Reinhold Kliegl
On Sun, Jul 18, 2010 at 12:28 PM, Reinhold Kliegl
<reinhold.kliegl at gmail.com> wrote:
> First of all, I really appreciate these efforts.
>
> Changing to R BLAS did not do the trick for me on Mac OS X (10.5.8).
> Here is the protocol.
>
> ###
> Last login: Sun Jul 18 12:04:10 on ttys000
> Reinhold:~ kliegl$ bash
>
> ~ > cd /Library/Frameworks/R.framework/Resources/lib
> /Library/Frameworks/R.framework/Resources/lib > ls
> i386 libRblas.vecLib.dylib libreadline.5.2.dylib
> libR.dylib libRlapack.dylib libreadline.dylib
> libRblas.0.dylib libgcc_s.1.dylib ppc
> libRblas.dylib libgfortran.2.dylib x86_64
>
> /Library/Frameworks/R.framework/Resources/lib > ln -sf
> libRblas.0.dylib libRblas.dylib
> /Library/Frameworks/R.framework/Resources/lib > R
>
> R version 2.11.1 Patched (2010-07-16 r52550)
> Copyright (C) 2010 The R Foundation for Statistical Computing
> ISBN 3-900051-07-0
>
> < clipped>
>
>> library(lme4)
>
> <clipped>
>
>> y <- (1:6)*pi # 3.14
>> x <- (1:6)^2
>> group <- gl(2,3)
>> for (i in 1:10) {
> + M1 <- lmer (y ~ x + (x | group))
> + M2 <- lmer (y ~ x + (x | group))
> + print(identical(ranef(M1),ranef(M2)))
> + print(identical(coef(M1),coef(M2)))
> + }
> [1] TRUE
> [1] TRUE
> [1] TRUE
> [1] TRUE
> [1] FALSE
> [1] FALSE
> [1] TRUE
> [1] TRUE
> [1] FALSE
> [1] FALSE
> [1] FALSE
> [1] FALSE
> [1] TRUE
> [1] TRUE
> [1] FALSE
> [1] FALSE
> [1] TRUE
> [1] TRUE
> [1] TRUE
> [1] TRUE
>
>> sessionInfo()
> R version 2.11.1 Patched (2010-07-16 r52550)
> i386-apple-darwin9.8.0
>
> locale:
> [1] de_DE/de_DE/C/C/de_DE/de_DE
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> other attached packages:
> [1] lme4_0.999375-34 Matrix_0.999375-41 lattice_0.18-8
>
> loaded via a namespace (and not attached):
> [1] grid_2.11.1 nlme_3.1-96 stats4_2.11.1
>>
> #####
>
> Did I do something wrong in getting the correct library? Is there a
> command to check whether R BLAS is used?
>
> Using require(lme4) instead of library(lme4) did not make a difference either.
>
> Many thanks again,
> Reinhold Kliegl
>
>
> On Sun, Jul 18, 2010 at 11:35 AM, Daniel Myall <daniel.lists at zeno.co.nz> wrote:
>> Hi Simon,
>>
>> The Apple vecLib BLAS appears to be the cause of the strangeness (at
>> least on my machine).
>>
>> Following:
>> http://cran.r-project.org/bin/macosx/RMacOSX-FAQ.html#Which-BLAS-is-used-and-how-can-it-be-changed_003f
>>
>>
>> I changed to the R BLAS and everything now works as it should (i.e., for
>> certain source data, lme4 now gives the same solution on multiple runs
>> instead of randomly giving one of two different "solutions" on each
>> run). The FAQ does note in relation to vecLib BLAS " Although fast, it
>> is not under our control and may possibly deliver inaccurate results."
>> which appears to be true, at least with lme4.
>>
>> What BLAS does the CRAN build machine use? If it does use the vecLib
>> BLAS is there a case for changing to R BLAS (maybe only when
>> building/checking lme4)?
>>
>> Cheers,
>> Daniel
>>
>> On 18/07/10 9:50 AM, Daniel Myall wrote:
>>> Hi Simon,
>>>
>>> Unfortunately I don't think is the full story (there are actual small
>>> differences in the fits).
>>>
>>> To match the lme4 tests the example should actually be:
>>>
>>> library(lme4)
>>> y<- (1:20)*pi
>>> x<- (1:20)^2
>>> group<- gl(2,10)
>>> for (i in 1:10) {
>>> M1<- lmer (y ~ x + ( x | group))
>>> M2<- lmer (y ~ x + ( x | group))
>>> print(identical(ranef(M1),ranef(M2)))
>>> print(ranef(M1)$group-ranef(M2)$group)
>>> }
>>>
>>> Which gives me (the number of TRUE/FALSE and the order change on each
>>> run, even if restarting R):
>>>
>>> [1] FALSE
>>> (Intercept) x
>>> 1 6.613450e-06 -6.898626e-08
>>> 2 -6.613462e-06 6.898637e-08
>>> [1] TRUE
>>> (Intercept) x
>>> 1 0 0
>>> 2 0 0
>>> [1] FALSE
>>> (Intercept) x
>>> 1 -6.613450e-06 6.898626e-08
>>> 2 6.613462e-06 -6.898637e-08
>>> [1] FALSE
>>> (Intercept) x
>>> 1 6.613450e-06 -6.898626e-08
>>> 2 -6.613462e-06 6.898637e-08
>>> [1] TRUE
>>> (Intercept) x
>>> 1 0 0
>>> 2 0 0
>>> [1] FALSE
>>> (Intercept) x
>>> 1 -6.613450e-06 6.898626e-08
>>> 2 6.613462e-06 -6.898637e-08
>>> [1] FALSE
>>> (Intercept) x
>>> 1 -6.613450e-06 6.898626e-08
>>> 2 6.613462e-06 -6.898637e-08
>>> [1] FALSE
>>> (Intercept) x
>>> 1 -6.613450e-06 6.898626e-08
>>> 2 6.613462e-06 -6.898637e-08
>>> [1] TRUE
>>> (Intercept) x
>>> 1 0 0
>>> 2 0 0
>>> [1] TRUE
>>> (Intercept) x
>>> 1 0 0
>>> 2 0 0
>>>
>>>
>>> Although only small differences, I assume lme4 should be deterministic?
>>>
>>> Cheers,
>>> Daniel
>>>
>>>
>>>
>>>
>>> On 18/07/10 8:23 AM, Simon Urbanek wrote:
>>>> Ok, I think I found the issue. I'm not sure why this varies by
>>>> platform but the mismatch is due to the @env slot. Two environments
>>>> are only identical if it is *the* same environment (i.e. the same
>>>> pointer). However, M1 and M2 have different environments. The content
>>>> of those environments is identical, but that is irrelevant as it's
>>>> not the same pointer. Hence identical(M1, M2) fails (and serialize
>>>> comparison succeeds as it cares only about the content).
>>>>
>>>> So the short story is don't use identical() to compare the models
>>>> (unless you remove @env first). The long story raises the question
>>>> whether identical() should really return FALSE for environments like
>>>>> identical(new.env(),new.env())
>>>> [1] FALSE
>>>> I can see arguments both ways but for the purposes of comparing
>>>> values there should be an option that the above is TRUE.
>>>>
>>>> To be honest I don't see why this has not shown up on other platforms
>>>> as that is a global issue... (I hope this is the full story - I
>>>> didn't try all the combinations to see if setting @env to the same
>>>> environment will appease identical() for all the models)
>>>>
>>>> Cheers,
>>>> Simon
>>>>
>>>>
>>>> On Jul 17, 2010, at 3:49 PM, Simon Urbanek wrote:
>>>>
>>>>> Daniel,
>>>>>
>>>>> thanks for the test case. I did run it in valgrind but nothing
>>>>> showed up, however ...
>>>>>
>>>>> I'm starting to have a suspicion that this has something to do with
>>>>> identical() - look at this:
>>>>>
>>>>>> identical(M1,M2)
>>>>> [1] FALSE
>>>>>> all(serialize(M1,NULL)==serialize(M2,NULL))
>>>>> [1] TRUE
>>>>>> identical(unserialize(serialize(M1,NULL)),unserialize(serialize(M2,NULL)))
>>>>>>
>>>>> [1] FALSE
>>>>>> identical(unserialize(serialize(M1,NULL)),unserialize(serialize(M1,NULL)))
>>>>>>
>>>>> [1] FALSE
>>>>>
>>>>> So I think this may be a bug in identical() mainly because of the
>>>>> last one. I'll need to take identical() apart to see where it fails
>>>>> ... I'm CCing this to R-devel as the current issue seems more like
>>>>> an R issue so more eyes can have a look ...
>>>>>
>>>>> Cheers,
>>>>> Simon
>>>>>
>>>>>
>>>>> [FWIW this is tested in today's R-devel (with valgrind level 2) on
>>>>> x86_64 OS X 10.6.4 with lme4 from CRAN and Matrix form R-devel
>>>>> Recommended]
>>>>>
>>>>>
>>>>> On Jul 17, 2010, at 4:50 AM, Daniel Myall wrote:
>>>>>
>>>>>> I've done some further testing (R 2.11.1) and the issue is not
>>>>>> limited to Leopard.
>>>>>>
>>>>>> Using the test:
>>>>>>
>>>>>> library(lme4)
>>>>>> y<- (1:20)*pi
>>>>>> x<- (1:20)^2
>>>>>> group<- gl(2,10)
>>>>>> for (i in 1:10) {
>>>>>> M1<- lmer (y ~ x + ( x | group))
>>>>>> M2<- lmer (y ~ x + ( x | group))
>>>>>> print(identical(M1,M2))
>>>>>> }
>>>>>>
>>>>>> For CRAN lme4 and Matrix:
>>>>>>
>>>>>> 32 bit on Leopard: R CMD check fails; different results (on most runs)
>>>>>> 32 bit on Snow Leopard: R CMD check passes; different results (on
>>>>>> some runs).
>>>>>> 64 bit on Snow Leopard: R CMD check passes; identical results
>>>>>>
>>>>>> For SVN version of Matrix with CRAN lme4:
>>>>>>
>>>>>> 32 bit on Snow Leopard: different results (on all runs).
>>>>>> 64 bit on Snow Leopard: different results (on all runs)
>>>>>>
>>>>>> For SVN version of Matrix with SVN lme4a:
>>>>>>
>>>>>> 32 bit on Snow Leopard: different results (on all runs).
>>>>>> 64 bit on Snow Leopard: identical results
>>>>>>
>>>>>> I couldn't reproduce on Linux 32/64bit. Is it time to jump into
>>>>>> valgrind to try and find the cause?
>>>>>>
>>>>>> Cheers,
>>>>>> Daniel
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 17/07/10 5:51 PM, John Maindonald wrote:
>>>>>>> In principle, maybe a Snow Leopard version might be posted
>>>>>>> as an alternative, if someone can provide one. But I take it
>>>>>>> that the issue is now a bit wider than tests that fail on Leopard
>>>>>>> vs passing on Snow Leopard?
>>>>>>>
>>>>>>
>>>>
>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>
More information about the R-sig-mixed-models
mailing list