[R-SIG-Mac] [R-sig-ME] lme4 missing from repositories?

jochen laubrock jochen.laubrock at gmail.com
Fri Oct 22 15:23:36 CEST 2010 

Interesting. 

When using R 2.12.0 in 32-Bit mode, I always get deterministic behavior with the reference BLAS, but random behavior with veclib. With R 2.11.1, I always get random behavior in 32-Bit mode, regardless of what BLAS implementation I chose. Finally, with R in 64-Bit mode, behavior is always deterministic (for both R 2.11.1 and R 2.12.0).

The following table summarizes the behavior (for details including sessionInfo() scroll down to end of post):

		R 2.11.1			R 2.12.0
		32-Bit	64-Bit	32-Bit	64-Bit
vecLib	rand	det		rand	det
RBLAS	rand	det		det		det

All of this has been tested on a MacPro4,1 (Quad-Core Intel Xeon, 2.26 GHz) and the 2.12.0 behavior confirmed on a MacBookPro5,5 (Intel Core 2 Duo, 2.53 GHz), with lme4_0.999375-35 and Matrix_0.999375-44.

Maybe there were two bugs, and one has been fixed in 2.12.0? Also, could it be that vecLib takes some numerical shortcuts that escalate? In that case, maybe the behavior should be brought to Apple's attention, using bugreport.apple.com. Finally, should the actual BLAS version used be included in sessionInfo()?

Thanks,
Jochen



On Oct 21, 2010, at 19:15 , Marc Schwartz wrote:

> Interesting. No matter what I do here, I can't seem to get the test to fail using R's BLAS with clean 32 bit builds. So perhaps it is not just the BLAS, but a combination of R's BLAS and specific hardware?, which gets me into a realm of knowledge below the event horizon. 
> 
> Have there been any repeatable scenarios where vecLib can be used without failure on a particular Mac platform?
> 
> Also, I just noted Simon's reply to a different thread on r-sig-mac to Stefan Evert, in which he notes that there may be a change in the default BLAS for OSX to vecLib in the next R release. Of course, now given Prof. Ripley's observations, it will be interesting to see the actual impact in the wild.
> 
> Thanks,
> 
> Marc
> 
> On Oct 21, 2010, at 11:23 AM, Prof Brian Ripley wrote:
> 
>> Let me point out https://stat.ethz.ch/pipermail/r-sig-mac/2010-July/007608.html
>> 
>> This is not just a BLAS issue: I saw it with both vecLib and the reference BLAS.
>> 
>> The lme4 code is doing exactly the same calculation for M2. and M2, but sometimes when it does that calculation the first time in a session it gives a different answer.  That makes it really hard to get a handle on, and easy to suppose one has a fix (been there a few times myself).
>> 
>> 
>> On Thu, 21 Oct 2010, Marc Schwartz wrote:
>> 
>>> 
>>> On Oct 21, 2010, at 8:47 AM, Federico Calboli wrote:
>>> 
>>>> Mark,
>>>>> 
>>>>> To the extent that it may be helpful here and I can do more if need be, I built 32 bit R 2.12.0 patched on Snow Leopard (10.6.4), using the R BLAS rather than Apple's veclib. This is on an early 2009 17" MBP with a 2.93 Ghz Core 2 Duo (MacBookPro5,2) and 4Gb of RAM.
>>>>> 
>>>>> Based upon Doug's comment in this thread that the issue may be related to the use of Apple's veclib BLAS, as opposed to R's reference BLAS, I ran some tests.
>>>>> 
>>>>> My config includes:
>>>>> 
>>>>> --without-blas --without-lapack
>>>>> 
>>>>> just to be sure that the above is the correct invocation, based upon what I found online.
>>>>> 
>>>>> Using this build, with all CRAN packages freshly installed using this build, I ran the example used here with lme4 0.999375-35. I get:
>>>>> 
>>>>> library(lme4)
>>>>> y <- (1:20)*pi; x <- (1:20)^2;group <- gl(2,10)
>>>>> M2. <- lmer (y ~ 1 + x + (1 + x | group))
>>>>> M2 <- lmer (y ~ x + ( x | group))
>>>>> 
>>>>>> identical(fixef(M2), fixef(M2.))
>>>>> [1] TRUE
>>>>> 
>>>>> 
>>>>> 
>>>>> I then created a function so that I could use replicate() to run this test a "larger" number of times:
>>>>> 
>>>>> testlme4 <- function()
>>>>> {
>>>>> y <- (1:20)*pi; x <- (1:20)^2;group <- gl(2,10)
>>>>> M2. <- lmer (y ~ 1 + x + (1 + x | group))
>>>>> M2 <- lmer (y ~ x + ( x | group))
>>>>> identical(fixef(M2), fixef(M2.))
>>>>> }
>>>>> 
>>>>> 
>>>>> RES <- replicate(1000, testlme4())
>>>>> 
>>>>>> all(RES)
>>>>> [1] TRUE
>>>>> 
>>>>>> table(RES)
>>>>> RES
>>>>> TRUE
>>>>> 1000
>>>>> 
>>>>> Does the example need to be run a "very large" number of times to be sure that it does not fail, or is the above a reasonable indication that the use of R's BLAS is a more appropriate default option for R on OSX?  If I am not mistaken (and somebody correct me if wrong), R's BLAS is the default on Windows and Linux (from my recollections on Fedora). Why should OSX be different in that regard?
>>>> 
>>>> Thanks for the very informative post. I added R-Mac in my reply to see if someone can come up with a response to your query. It would also be interesting to know if it were possible to switch the OSX R binary to use the R BLAS library.
>>>>> 
>>>>> Also, as an aside to Federico, I use 32 bit R on OSX largely because I have to interact with an Oracle server via RODBC. The only ODBC drivers available for Oracle on OSX are 32 bit and they are not compatible with 64 bit R. It would be rather cumbersome when running reports (via Sweave) to first extract the data in 32 bit R and then switch to 64 bit R to run the reports. I can run it all in a single step using 32 bit R. I also do not have a need for the larger memory address space afforded by 64 bit R.
>>>> 
>>>> I'm very primitive in any integration between R and anything else, so much so that I abandoned Emacs (well integrated with R) for Vim (not as well integrated). On the other hand I do need the greater memory address space of R64. I understand my needs and habits are not universally shared, but, if the *only* reason for using R32 vs R64 is the 20% speed difference, I'd use R64 for running lme4.
>>> 
>>> 
>>> 
>>> OK, so here is some more data.  I wondered if my build using R's BLAS may have possibly been a Type II error. So I re-built 32 bit R (same SVN checkout code) using:
>>> 
>>> --with-blas='-framework vecLib' --with-lapack
>>> 
>>> I then completely removed my old R build (using R's BLAS) and the installed CRAN packages. I re-installed R using the above configuration and then cleanly re-installed the required CRAN packages.
>>> 
>>> Here are the results:
>>> 
>>> y <- (1:20)*pi; x <- (1:20)^2;group <- gl(2,10)
>>> M2. <- lmer (y ~ 1 + x + (1 + x | group))
>>> M2 <- lmer (y ~ x + ( x | group))
>>> 
>>>> identical(fixef(M2), fixef(M2.))
>>> [1] FALSE
>>> 
>>> testlme4 <- function()
>>> {
>>> y <- (1:20)*pi; x <- (1:20)^2;group <- gl(2,10)
>>> M2. <- lmer (y ~ 1 + x + (1 + x | group))
>>> M2 <- lmer (y ~ x + ( x | group))
>>> identical(fixef(M2), fixef(M2.))
>>> }
>>> 
>>> RES <- replicate(1000, testlme4())
>>> 
>>>> all(RES)
>>> [1] FALSE
>>> 
>>>> table(RES)
>>> RES
>>> FALSE  TRUE
>>> 496   504
>>> 
>>> So the test in question seems to fail with P(Failure) ~ 0.5 using Apple's veclib BLAS.
>>> 
>>> I should also note that all of my testing is from the CLI using the OSX terminal. I do not use R.app.
>>> 
>>> In response to Huang's reply, regarding his use of the shared lib approach, I wonder if there is some other interaction going on, either in the BLAS libs, or perhaps in the installed version of lme4, when one BLAS versus the other is in use at the time of lme4 installation, since it is installed from source on OSX. Note that I used a fully clean build of R and the required CRAN packages for each set of tests.
>>> 
>>> If there is some other testing that I can do, let me know. But the above results with a clean build of R and lme4 each time, would seem to further reinforce a reconsideration of the use of Apple's veclib BLAS as the default for CRAN binary builds of R on OSX.
>>> 
>>> Regards,
>>> 
>>> Marc
>>> 
>>> _______________________________________________
>>> R-SIG-Mac mailing list
>>> R-SIG-Mac at stat.math.ethz.ch
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>> 
>> 
>> -- 
>> Brian D. Ripley,                  ripley at stats.ox.ac.uk
>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>> University of Oxford,             Tel:  +44 1865 272861 (self)
>> 1 South Parks Road,                     +44 1865 272866 (PA)
>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>> 
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac at stat.math.ethz.ch
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> 
> _______________________________________________
> R-SIG-Mac mailing list
> R-SIG-Mac at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac





 ===========================================
 = libRblas.dylib -> libRblas.vecLib.dylib =
 ===========================================
 ==========
 = 32-Bit =
 ==========

------ R 2.11.1 ------
> table(RES)
RES
FALSE  TRUE 
   44    56 
> sessionInfo()
R version 2.11.1 (2010-05-31) 
i386-apple-darwin9.8.0 

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_0.999375-35   Matrix_0.999375-44 lattice_0.18-8    

loaded via a namespace (and not attached):
[1] grid_2.11.1   nlme_3.1-96   stats4_2.11.1

------ R 2.12.0 ------
> table(RES)
RES
FALSE  TRUE 
   52    48

> sessionInfo()
R version 2.12.0 Patched (2010-10-21 r53386)
Platform: i386-apple-darwin9.8.0/i386 (32-bit)

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_0.999375-35   Matrix_0.999375-44 lattice_0.19-13   

loaded via a namespace (and not attached):
[1] grid_2.12.0   nlme_3.1-97   stats4_2.12.0 tcltk_2.12.0  tools_2.12.0 


 ======================================
 = libRblas.dylib -> libRblas.0.dylib =
 ======================================
 ==========
 = 32-Bit =
 ==========
------ R 2.11.1 ------
> table(RES)
RES
FALSE  TRUE 
   59    41

> sessionInfo()
R version 2.11.1 (2010-05-31) 
i386-apple-darwin9.8.0 

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_0.999375-35   Matrix_0.999375-44 lattice_0.18-8    

loaded via a namespace (and not attached):
[1] grid_2.11.1   nlme_3.1-96   stats4_2.11.1 tcltk_2.11.1  tools_2.11.1

------ R 2.12.0 ------
> table(RES)
RES
TRUE 
 100 

> sessionInfo()
R version 2.12.0 Patched (2010-10-21 r53386)
Platform: i386-apple-darwin9.8.0/i386 (32-bit)

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_0.999375-35   Matrix_0.999375-44 lattice_0.19-13   

loaded via a namespace (and not attached):
[1] grid_2.12.0   nlme_3.1-97   stats4_2.12.0


 ==========
 = 64-Bit =
 ==========
------ R 2.11.1 ------
> table(RES)
RES
TRUE 
 100 

> sessionInfo()
R version 2.11.1 (2010-05-31) 
x86_64-apple-darwin9.8.0 

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_0.999375-33   Matrix_0.999375-44 lattice_0.18-8    

loaded via a namespace (and not attached):
[1] grid_2.11.1 nlme_3.1-96


------ R 2.12.0 ------
> table(RES)
RES
TRUE 
 100

> sessionInfo()
R version 2.12.0 Patched (2010-10-21 r53386)
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_0.999375-35   Matrix_0.999375-44 lattice_0.19-13   

loaded via a namespace (and not attached):
[1] grid_2.12.0   nlme_3.1-97   stats4_2.12.0


----
Jochen Laubrock, Dept. of Psychology, University of Potsdam,
Karl-Liebknecht-Strasse 24-25, 14476 Potsdam, Germany
phone: +49-331-977-2346, fax: +49-331-977-2793

More information about the R-SIG-Mac mailing list