[R-sig-ME] Resume terminated lmer fit if verbose=TRUE?

[Ben Bolker]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/27/2011 02:39 PM, Mike Lawrence wrote:
> Thanks to Harold for pointing out the start argument and indicating
> that the verbose output can be supplied as its value. Thanks also to
> both Harold and Ben for warning on over-parameterization, though it
> may not fully apply here as my model is of the form "y ~ (1|random) +
> V1*V2*V3*V4*V5*V6*V7". (well, I might have a priori reason to
> eliminate one or two fixed effects...)
> 
> Mike

  OK.

  How many fixed effect parameters total?
(i.e.,  ncol(model.matrix(~V1*V2*V3*V4*V5*V6*V7,data)) ...)
  How many observations?


> 
> On Wed, Jul 27, 2011 at 10:47 AM, Ben Bolker <bbolker at gmail.com> wrote:
> On 07/27/2011 09:29 AM, Doran, Harold wrote:
>>>> Yes. See ?lmer and the start argument. You can provide the function
>>>> with starting values, which can come as the last iteration of the
>>>> output from verbose. 100 hours is a ridiculous amount of computing
>>>> time.
> 
>  Oops. I was wrong (thanks).  I was thinking of lme.
> 
>  A common theme on this list seems to be that people set up models of
> the form
> 
> response ~ (a lot of fixed effects) + (a lot of fixed effects|grouping)
> 
>  The problem here is that n fixed effects  interacting with the
> grouping variable means n*(n+1) variance-covariance parameters, which is
> often slow to fit ... see e.g.
> http://article.gmane.org/gmane.comp.lang.r.lme4.devel/6308 ...
> 
> 
>>>>
>>>> You might want to check your model and make sure it isn't
>>>> overparameterized.
>>>>
>>>>> -----Original Message----- From:
>>>>> r-sig-mixed-models-bounces at r-project.org
>>>>> [mailto:r-sig-mixed-models- bounces at r-project.org] On Behalf Of
>>>>> Mike Lawrence Sent: Wednesday, July 27, 2011 9:16 AM To:
>>>>> r-sig-mixed-models at r-project.org Subject: [R-sig-ME] Resume
>>>>> terminated lmer fit if verbose=TRUE?
>>>>>
>>>>> Hi folks,
>>>>>
>>>>> I ran a binomial lmer on a large data set with lots of levels of a
>>>>> random effect (1000+) and a large fixed effects structure (7
>>>>> variables all interacting). My local machine wasn't up to the task
>>>>> (it needs about 16GB of memory so far as I can tell), so I put it
>>>>> on a serial node on a supercomputer to which I have access. I asked
>>>>> for 100 hours of compute time, but it seems that the model went
>>>>> over time and the process was terminated by the system's queue.
>>>>> However, I ran it in verbose mode and have all the output, so I'm
>>>>> wondering if there's any way to use this information to resubmit
>>>>> the job and have it resume where it left off. Any ideas?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Mike
>>>>>
>>>>> -- Mike Lawrence Graduate Student Department of Psychology
>>>>> Dalhousie University
>>>>>
>>>>> Looking to arrange a meeting? Check my public calendar:
>>>>> http://tr.im/mikes_public_calendar
>>>>>
>>>>> ~ Certainty is folly... I think. ~
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> 
>>
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>

> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4wbagACgkQc5UpGjwzenP9UgCghn9jvUzFh8TmpemVvcoAfgBU
iaIAn2OKmygNQh6CAn6FV379CmG8Rgx0
=6VoE
-----END PGP SIGNATURE-----