[R-sig-ME] I'm sorry, and here is what I mean to ask about speed

[Ken Beath]

On Thu, September 24, 2009 1:42 am, Paul Johnson wrote:
> Do you have a copy of HLM6 on your system?  Maybe you could help me by
> running the same model in R (with any of the packages such as lme4,
> nlme, or whatever) and HLM6 and let us all know if you get similar
> estimates and how long it takes to run each one.
>

There is a time-limited trial version of HLM
http://www.ssicentral.com/hlm/downloads.html

I haven't tried it, but I expect that I wont like it. I expect it will
have a philosophy that every model is special, rather than providing a
general model specification as in R.

>
> My colleague has HLM6 on Windows XP and he compared a two-level linear
> mixed effects model fitted with lmer from lme4 against HLM6.  He
> surprised my by claiming that the HLM6 model estimation was completed
> in about 1.5 seconds and the lmer estimation took 50 seconds.  That
> did not seem right to me.  I looked a bit at his example and made a
> few mental notes so I could ask you what to look for when I go back to
> dig into this.  There are 27000 cases in his datasets and he has about
> 25 variables at the lower level of observation and 4 or 5 variables at
> the higher level, which I think is the county of survey respondents.
> He is fitting a random intercept (random across counties) and several
> random slopes for the higher level variables.
>
> He pointed out that the mlWin website reported speed differences in
> 2006 that were about the same.  Of course, you and I know that R and
> all of the mixed effects packages have improved significantly since
> then. That is why the speed gap on the one Windows XP system surprised
> me.
>
> Can you tell me if you see a difference between the two programs (if
> you have HLM6).  If you see a difference on the same magnitude, it may
> mean we are not mistaken in our conclusion.  But if you see no
> difference, then it will mean I'm getting it wrong and I should
> investigate more. If I can't solve it, I should provide a reproducible
> example for your inspection.  I will ask permission to release the
> private data to  you in that case.
>
> Perhaps you think there are good reasons why R estimation takes longer.
> E.g.:
> 1. HLM programmers have full access to benefit from optimizations in
> lmer and other open programs, but they don't share their optimizations
> in return.
> 2. lmer and other R routines are making calculations in a better way,
> a more accurate way, so we should not worry that they take longer.

I don't know what HLM use. lme used an EM algorithm which is one of the
slower ways, but with excellent properties, and I assume lmer uses the
same. It may be that Doug has set some of the convergence criteria so that
it will work with very complex models and these could be relaxed at the
users peril.  I would prefer slower.  R can also take a lot of time to do
some things, and the only way around this is to rewrite everything in C or
Fortran.

To me R has the advantage that I can set up a large number of complex
analyses with graphs, and just run the lot. If time became a concern
because it was preventing other use of my computer then I would set up a
linux server and run everything remotely.

>    That was my first guess, in the original mail I said I thought that
> HLM was using PQL whereas lmer is using Laplace or Adaptive Gaussian
> Quadrature.  But Doug's comment indicated that I was mistaken to
> expect a difference there because REML is the default in lmer and it
> is also what HLM is doing, and there's no involvement of quadrature or
> integral approximation in a mixed linear model (gaussian dependent
> variable).
>
> On the other hand, perhaps you are (like me) surprised by this
> difference and you want to help me figure out the cause of the
> differences.  If you have ideas about that, maybe we can work together
> (I don't suck at C!). I have pretty much experience profiling programs
> in C and did some optimization help on a big-ish C++ based R package
> this summer.
>
> So far, I have a simple observer's interest in this question.   I
> advise people whether it is beneficial for them to spend their scarce
> resources on a commercial package like HLM6 and one of the factors
> that is important to them is how "fast" the programs are.   I
> personally don't see an urgent reason to buy HLM because it can
> estimate a model in 1 second and an open source approach requires 50
> seconds.  But I'm not the one making the decision. If I can make the R
> version run almost as fast as HLM6, or provide reasons why people
> might benefit from using a program that takes longer, then I can do my
> job of advising the users.
>
> I am sorry if this question appears impertinent or insulting. I do not
> mean it as a criticism.
>
> --
> Paul E. Johnson
> Professor, Political Science
> 1541 Lilac Lane, Room 504
> University of Kansas
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>