[R-sig-ME] Improving computation time for a binary outcome in lme4

[Adam D. I. Kramer]

On Mon, 24 May 2010, Douglas Bates wrote:

> If you use the verbose option to lmer and to glmer on similar problems you
> will see that lmer if optimizing over fewer parameters than is glmer.

Just one potentially useful observation: Turning on "verbose" makes the
waiting period much MUCH more tolerable. It's kinda like a progress bar--you
know glmer is doing something and that makes it easier to wait.

For some huge models with bigger-than-I-needed data sets (back in the
netflix prize days), I just let R run overnight and got what I wanted--but I
had never let it go more than an hour before I worried that it was looping.

>> Are there any suggestions on what I can do (other than simplify the model)
>> to improve the computation time for a binary outcome?
>
> There are the usual suspects of getting access to a fast computer with
> lots of memory and a 64-bit operating system.  You could see whether an
> accelerated BLAS will help.  For example, Revolution R has the MKL BLAS
> built-in.  Regrettably, that isn't always a speed boost.  We have seen
> situations where multi-threaded BLAS actually slow down sparse matrix
> operations because the communications overhead is greater than the time
> savings of being able to perform more flops per second.

My kingdom for multi-threaded nlm()...

--Adam