[R-sig-ME] Large data set and mixed models
bates at stat.wisc.edu
Fri Oct 17 17:15:20 CEST 2008
On Fri, Oct 17, 2008 at 8:34 AM, Rense Nieuwenhuis
<rense.nieuwenhuis at me.com> wrote:
> Dear Michael,
> perhaps you could send more details about the model you're trying to
> estimate, so we could be of help.
> - What is the model specification
> - What happens -> error message, uninterpretable findings?
> - A closer description of the data
> - What system are you trying to estimate this model with?
> In general I wouldn't say the 100000 cases is 'huge' in terms of R-Project.
> Sure, some models will take some time to converge, but it should be doable.
Agreed. The largest example that I have fit with lme4 in R has about
1.7 million observations and over 60,000 non-nested random effects.
> If you'd send me (a sample of) your data, I'd be willing to take a look at
A good start would be if Michael could show us a transcript of his
attempt to fit the model he want in R, including the output from
so we know the versions of all packages being used.
> On 17-okt-2008, at 15:02, Michael Beaulieu wrote:
>> I would like two compare the diving behaviour of two groups of penguins (7
>> penguin in each group). Each penguin performed several dives within several
>> foraging trips. As a result, I got a huge data set of dives (nearly 100000).
>> To compare the diving behaviour of the two groups, I used a mixed model
>> -the penguin as a random factor,
>> -the number of dives nested in the foraging trip as a repeated factor,
>> -the group, the foraging trip and maximal depth as fixed factors.
>> Covariance structure was auto-regressive.
>> I tried this model on SPSS, SAS and R but all failed.
>> Has anybody been faced with such a huge dataset analysed with mixed
>> Thank you
>> R-sig-mixed-models at r-project.org mailing list
> R-sig-mixed-models at r-project.org mailing list
More information about the R-sig-mixed-models