[R-sig-ME] More naive questions: Speed comparisons? what is a "stack imbalance" in lmer? does lmer center variables?

Wed Sep 23 19:37:15 CEST 2009

On Wed, Sep 23, 2009 at 12:21 PM, Ben Bolker <bolker at ufl.edu> wrote:
>  Source *is* available:

Thanks for the correction, Ben.  I withdraw my comments about ADMB and
welcome it to the Open Source software arena.

> Peter Dalgaard wrote:
>> Ben Bolker wrote:
>>>    As of sometime last year (? I think ?), ADMB is
>>> free/gratis/libre/open source (BSD licensed).
>>
>> As of _next_ year is more like it. Their downloads are still
>> binary-only. BSD does not imply Open Source when you don't have the
>> sources, and "open source project" in this case means a project to make
>> the software open source. The intention appears to be honourable, though.
>>
>>   Even before that,
>>> glmmADMB (which was an R package with a binary component, available for
>>> download) was "free as in beer". In its current status, I think of ADMB
>>> in the same category as WinBUGS -- a powerful, albeit sometimes
>>> unwieldy, tool that can be used through an R interface to solve general
>>> problems by writing model descriptions in a non-R language.
>>>   I have to agree with Kevin that the diversity of mixed model software
>>> tools is a good thing.
>>>
>>>   cheers
>>>     Ben Bolker
>>>
>>> Douglas Bates wrote:
>>>> Got to disagree with you, Kevin.  admb and asreml are not part of R,
>>>> even in the general sense of R packages.  R is Open Source - they are
>>>> not. Tacking on an R interface to proprietary software and saying it
>>>> is available in R is misleading and dishonest.
>>>>
>>>> On Wed, Sep 23, 2009 at 8:54 AM, Kevin Wright <kw.stat at gmail.com> wrote:
>>>>> Paul,
>>>>>
>>>>> It appears to me that the published timings you reference are
>>>>> comparing the __nlme__ package with other software.  So the answer is
>>>>> yes, nlme really is that slow for some models.  You are probably aware
>>>>> that the __lme4__ package has faster algorithms.
>>>>>
>>>>> There are many ways to fit mixed models in R including nlme, lme4,
>>>>> MCMCglmm, admb asreml, BUGS, etc.  If I was teaching a course, I would
>>>>> try to expose students to at least two of those in some detail and
>>>>> touch briefly on the others: nlme can fit a variety of complex
>>>>> varaiance structures, lme4 has faster algorithms, asreml is the only
>>>>> choice of animal/plant breeders and has commercial support, MCMCglmm
>>>>> has some Bayesian aspects and can fit some heteroskedastic variance
>>>>> structures, admb is used in Fish & Wildlife, etc.
>>>>>
>>>>> Mixed model fitting in R is definitely not a case of "one size fits all".
>>>>>
>>>>> Kevin Wright
>>>>>
>>>>>
>>>>> On Wed, Sep 23, 2009 at 1:36 AM, Paul Johnson <pauljohn32 at gmail.com> wrote:
>>>>>> Sent this to r-sig-debian by mistake the first time.  Depressing.
>>>>>>
>>>>>> 1.  One general question for general discussion:
>>>>>>
>>>>>> Is HLM6 faster than lmer? If so, why?
>>>>>>
>>>>>> I'm always advocating R to students, but some faculty members are
>>>>>> skeptical.  A colleague compared the commercial HLM6 software to lmer.
>>>>>>  HLM6 seems to fit the model in 1 second, but lmer takes 60 seconds.
>>>>>>
>>>>>> If you have HLM6 (I don't), can you tell me if you see similar differences?
>>>>>>
>>>>>> My first thought was that LM6 uses PQL by default, and it would be
>>>>>> faster.  However, in the output, HLM6 says:
>>>>>>
>>>>>> Method of estimation: restricted maximum likelihood
>>>>>>
>>>>>> But that doesn't tell me what quadrature approach they use, does it?
>>>>>>
>>>>>> Another explanation for the difference in time might be the way HLM6
>>>>>> saves the results of some matrix calculations and re-uses them behind
>>>>>> the scenes.  If every call to lmer is re-calculating some big matrix
>>>>>> results, I suppose that could explain it.
>>>>>>
>>>>>> There are comparisons from 2006 here
>>>>>>
>>>>>> http://www.cmm.bristol.ac.uk/learning-training/multilevel-m-software/tables.shtml
>>>>>>
>>>>>> that indicate that lme was much slower than HLM, but that doesn't help
>>>>>> me understand *why* there is a difference.
>>>>>>
>>>>>> 2. What does "stack imbalance in .Call" mean in lmer?
>>>>>>
>>>>>> Here's why I ask.  Searching for comparisons of lmer and HLM,  I went
>>>>>> to CRAN &  I checked this document:
>>>>>>
>>>>>> http://cran.r-project.org/web/packages/mlmRev/vignettes/MlmSoftRev.pdf
>>>>>>
>>>>>> I *think* these things are automatically generated.  The version
>>>>>> that's up there at this moment  (mlmRev edition 0.99875-1)  has pages
>>>>>> full of the error message:
>>>>>>
>>>>>> stack imbalance in .Call,
>>>>>>
>>>>>> Were those always there?  I don't think so.   What do they mean?
>>>>>>
>>>>>> 3. In the HLM6 output, there is a message at the end of the variable list:
>>>>>>
>>>>>> '%' - This level-1 predictor has been centered around its grand mean.
>>>>>> '$' - This level-2 predictor has been centered around its grand mean.
>>>>>>
>>>>>> What effect does that have on the estimates?  I believe it should have
>>>>>> no effect on the fixed effect slope estimates, but it seems to me the
>>>>>> estimates of the variances of random parameters would be
>>>>>> changed.  In order to make the estimates from lmer as directly
>>>>>> comparable as possible, should I manually center all of the variables
>>>>>> before fitting the model?   I'm a little stumped on how to center a
>>>>>> multi-category factor before feeding it to lmer.  Know what I mean?
>>>>>>
>>>>>> pj
>>>>>>
>>>>>> --
>>>>>> Paul E. Johnson
>>>>>> Professor, Political Science
>>>>>> 1541 Lilac Lane, Room 504
>>>>>> University of Kansas
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>>>
>>>>> _______________________________________________
>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>
>>
>
>
> --
> Ben Bolker
> Associate professor, Biology Dep't, Univ. of Florida
> bolker at ufl.edu / www.zoology.ufl.edu/bolker
> GPG key: www.zoology.ufl.edu/bolker/benbolker-publickey.asc
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>