As indicated by other respondents on this thread, just having multiple
processors or multiple computational cores on a single processor does not,
by itself, allow for faster execution of an R function like lmer.

There are many different factors that affect the speed with which
mixed-effects models can be fit.  It is true that the combination of a
large data set and a complex model can stress the capabilities of R and I
feel this will always be the case.

As some of the readers of this list are aware, during the last year much of
my development effort has shifted from R to the Julia language (
http://julialang.org).  I felt that the design of Julia would allow for
easier and more effective coding for models such a mixed-effects models and
for simulation-based methods such as Markov chain Monte Carlo.

It is a long process to start building capabilities into a new language - a
lot of the infrastructure must be rebuilt and this involves more than
simple translation of syntax.  Also language design is often a matter of
trial-and-error; you try something that you think should work and later
realize that there are better ways of doing it.  This means that early
adopters must put up with a rapidly changing structure in which what worked
a few weeks ago may no longer work.

As one of the early adopters I feel that Julia is fulfilling its promise
and that it will have a an impact comparable to that of R on the
statistical computing community.  This does not mean that it is trivial to
migrate from R to Julia.  At present Julia is still difficult to install
and maintain, the package system is still not well fleshed out, the
graphics packages are still comparatively primitive, ...  And the function
names and the overall style of programming are different than in R.  I have
been using S and R for about 30 years and usually get by without referring
to manuals to find out what function does ...  It is not altogether
pleasant to experience again the frustration of needing to look up function
names over and over.

This is all to say that Julia is not a "finished" system the way that R is.
 However, it is fast and more capable than R, even in its current state.
 And it can handle larger data sets because of some fundamental aspects of
the language design.

I have recently released a MixedModels package for Julia.  At present it
only fits linear mixed-effects models with scalar random-effects terms but
other capabilities will be added soon.

Because this is a mailing list about mixed-effects models in R I think it
would be inappropriate for me to use it to go into details about Julia.
 There is a Google groups mailing list called julia-stats which is where
such discussion should take place.



On Thu, Apr 18, 2013 at 12:33 AM, Gabriel Baud-Bovy <
baud-bovy.gabriel@hsr.it> wrote:

> Another idea might be to use parallellization for bootsrap.
> Best,
> Gabriel
>
>
> On 4/18/2013 12:40 AM, Titus von der Malsburg wrote:
>
>>
>> Packages like parallel allow you to create temporary copies the current
>> R process such that the various copies can work on different problems
>> (using differnt cores).  Technically, that's similar to running several
>> instances of R in parallel, just more convenient because R automatically
>> collects the results of the parallel computations in the parent process.
>>
>> As you suspected, the execution of a single function like lmer can't be
>> split up with these packages.  Automatic parallelization of code written
>> for a single execution thread is a very hard problem.  In many cases
>> it's even impossible.  Therefore, in order to make use of several cores,
>> lmer would have to be rewritten in some way, which may also not be
>> trivial.  So, I'm afraid it's currently not possible to use several
>> cores and it's very likely going to stay that way for at least some
>> time.
>>
>> For now, I can offer only two ideas: 1.) If you have to run several
>> models, e.g. for different dependent variables, you can use parallel to
>> fit these models concurrently, each on one core.  2.) If your models are
>> fit on very large data sets, it may happen that your computer is running
>> out of RAM.  In this case, the operating system will extend the RAM
>> using disk space.  The problem with that is that hard disks are several
>> orders of magnitude slower than RAM and therefore everything will slow
>> down to a crawl.  The solution is then to extend the RAM of your
>> computer.  RAM is cheap but you have to make sure that you have a 64 bit
>> operating system.  32 bit operating systems can't make use of RAM
>> capacities larger than 4 GB.  There is probably an upper limit to how
>> much RAM R can use but I don't know that from the top of my head.  If
>> your problem really is RAM, then extending RAM will give you a
>> tremendous speed-up.
>>
>> Good luck!
>>
>>    Titus
>>
>> Felipe Vargas Reeve writes:
>>
>>>     Hi everyone, I want to know if somebody can help me with this: A
>>> methodology to increase the performance of R to achieve the convergence
>>> in
>>> lmer or nlme moldels.
>>>
>>>     Actually even if the computer presents more than one core I have read
>>> that R works with only one of them. Also I have read about the existence
>>> of
>>> packages (Eg. parallel) that can improve the speed of the computer. This
>>> is
>>> based in using all the cores of the pc, but I think this works for
>>> independent functions and it does not optimize only one process like in
>>> the
>>> case of the linear mixed model.
>>>
>>> Thanks for your help guys,
>>>
>>> Regards
>>> Felipe.
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> ______________________________**_________________
>>> R-sig-mixed-models@r-project.**org <R-sig-mixed-models@r-project.org>mailing list
>>> https://stat.ethz.ch/mailman/**listinfo/r-sig-mixed-models<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
>>>
>>
>>
>
> --
> ------------------------------**------------------------------**---------
> Gabriel Baud-Bovy               tel.: (+39) 02 2643 4839 (office)
> UHSR University                       (+39) 02 2643 3429 (laboratory)
> via Olgettina, 58                     (+39) 02 2643 4891 (secretary)
> 20132 Milan, Italy               fax: (+39) 02 2643 4892
>
>
> ______________________________**_________________
> R-sig-mixed-models@r-project.**org <R-sig-mixed-models@r-project.org>mailing list
> https://stat.ethz.ch/mailman/**listinfo/r-sig-mixed-models<https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
>

	[[alternative HTML version deleted]]

