[R-sig-ME] How to use all the cores while running glmer on a piecewise exponential survival with

[D. Rizopoulos]

As suggested, an approach could be to split the original big sample in manageable pieces, do the analysis in each, and then combine the results.

Geert Molenberghs, Geert Verbeke and colleagues have worked on this; a relevant recent papers seems to be: https://lirias2repo.kuleuven.be/bitstream/id/470902/

I hope it helps.

Best,
Dimitris


From: Ben Bolker <bbolker using gmail.com<mailto:bbolker using gmail.com>>
Date: Thursday, 23 Aug 2018, 10:29 PM
To: r-sig-mixed-models using r-project.org <r-sig-mixed-models using r-project.org<mailto:r-sig-mixed-models using r-project.org>>
Subject: Re: [R-sig-ME] How to use all the cores while running glmer on a piecewise exponential survival with


  Are the frequentist methods *not* faster?  I'd be pretty surprised,
unless some you're hitting some terrible memory bottleneck or something.


On 2018-08-23 03:30 PM, Adam Mills-Campisi wrote:
> We originally tried to use stan to estimate the model, we were getting
> performance issues. I assumed that the frequentist approaches would be
> faster.
>
> On Thu, Aug 23, 2018 at 12:28 PM Doran, Harold <HDoran using air.org> wrote:
>
>> No. You can change to an improved BLAS or I have found the Microsoft R has
>> some built in multithreading that is fast for matrix algebra and it passes
>> that benefit to lmer. From some experience, you can improve computational
>> time of an lmer model with Microsoft R
>>
>> -----Original Message-----
>> From: R-sig-mixed-models <r-sig-mixed-models-bounces using r-project.org> On
>> Behalf Of Adam Mills-Campisi
>> Sent: Thursday, August 23, 2018 3:18 PM
>> To: r-sig-mixed-models using r-project.org
>> Subject: [R-sig-ME] How to use all the cores while running glmer on a
>> piecewise exponential survival with
>>
>> I am estimating a piecewise exponential, mixed-effects, survival model
>> with recurrent events. Each individual in the dataset gets an individual
>> interpret (where using a PWP approach). Our full dataset has 10 million
>> individuals, with 180 million events. I am not sure that there is any
>> framework which can accommodate data at that size, so we are going to
>> sample. Our final sample size largely depends on how quickly we can
>> estimate the model, which brings me to my question: Is there a way to
>> mutli-thread/core the model? I tried to find some kind of instruction on
>> the web and the best lead I could find was a reference to this list serve.
>> Any help would be greatly appreciated.
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-models using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>>
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>

_______________________________________________
R-sig-mixed-models using r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

	[[alternative HTML version deleted]]