[R-sig-ME] How can I optimize the performance of mixed models?

[Titus von der Malsburg]

Packages like parallel allow you to create temporary copies the current
R process such that the various copies can work on different problems
(using differnt cores).  Technically, that's similar to running several
instances of R in parallel, just more convenient because R automatically
collects the results of the parallel computations in the parent process. 

As you suspected, the execution of a single function like lmer can't be
split up with these packages.  Automatic parallelization of code written
for a single execution thread is a very hard problem.  In many cases
it's even impossible.  Therefore, in order to make use of several cores,
lmer would have to be rewritten in some way, which may also not be
trivial.  So, I'm afraid it's currently not possible to use several
cores and it's very likely going to stay that way for at least some
time.

For now, I can offer only two ideas: 1.) If you have to run several
models, e.g. for different dependent variables, you can use parallel to
fit these models concurrently, each on one core.  2.) If your models are
fit on very large data sets, it may happen that your computer is running
out of RAM.  In this case, the operating system will extend the RAM
using disk space.  The problem with that is that hard disks are several
orders of magnitude slower than RAM and therefore everything will slow
down to a crawl.  The solution is then to extend the RAM of your
computer.  RAM is cheap but you have to make sure that you have a 64 bit
operating system.  32 bit operating systems can't make use of RAM
capacities larger than 4 GB.  There is probably an upper limit to how
much RAM R can use but I don't know that from the top of my head.  If
your problem really is RAM, then extending RAM will give you a
tremendous speed-up.

Good luck!

  Titus

Felipe Vargas Reeve writes:
>    Hi everyone, I want to know if somebody can help me with this: A
> methodology to increase the performance of R to achieve the convergence in
> lmer or nlme moldels.
>
>    Actually even if the computer presents more than one core I have read
> that R works with only one of them. Also I have read about the existence of
> packages (Eg. parallel) that can improve the speed of the computer. This is
> based in using all the cores of the pc, but I think this works for
> independent functions and it does not optimize only one process like in the
> case of the linear mixed model.
>
> Thanks for your help guys,
>
> Regards
> Felipe.
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


-- 
Dr. Titus von der Malsburg
DFG Research Group 868: Mind and Brain Dynamics
Dept. of Linguistics, University of Potsdam
http://www.ling.uni-potsdam.de/~malsburg/