[R-sig-ME] [R] degrees of freedom and random effects in lmer

[Douglas Bates]

I also apologize because I sent an incomplete reply.  I hit the "send"
key sequence before I planned to.

I was going to say that it is not entirely clear exactly what one
should regard as "the degrees of freedom" for random effects terms.
Fixed-effects models have a solid geometric interpretation that gives
an unambiguous definition of degrees of freedom.  Models with random
effects don't have nearly the same clarity.

If one counts parameters to be estimated then random effects for the
levels of a factor cost only 1 degree of freedom, regardless of the
number of levels.  This is the lowest number one could imagine for the
degrees of freedom and, if you regard degrees of freedom as measuring
the explanatory power of a model, this can be a severe underestimate.

If one goes with the geometric argument and measures something like
the dimension of the predictor space then the degrees of freedom would
be the number of levels or that number minus 1, which is what you were
assuming.  This is the same as counting the number of coefficients in
the linear predictor.  The problem here is the predictor doesn't have
all of the degrees of freedom associated with the geometric subspace.
The "estimates" of the random effects are not the solution of a least
squares problem.  They are the solution of a penalized least squares
problem and the penalty has a damping effect on the coefficients.

An argument can be made that the effective degrees of freedom lies
between these two extremes and can be measured according to the trace
of the "hat" matrix.

I really don't know what the best answer is.  In a way I think it is
best to avoid trying to force a definition of degrees of freedom in
mixed models.


On Jan 16, 2008 12:17 PM, Feldman, Tracy <tsfeldman at noble.org> wrote:
> Dear Dr. Bates,
>
> Thank you for your response.  Also, I did not intend to make value
> judgements in my questions below.  I had simply misunderstood what was
> the parameter in the Likelihood Ratio Test (and I assumed that I knew).
> I apologize.
>
> Take care,
> Tracy
>
>
> -----Original Message-----
> From: dmbates at gmail.com [mailto:dmbates at gmail.com] On Behalf Of Douglas
> Bates
> Sent: Wednesday, January 16, 2008 11:59 AM
> To: Feldman, Tracy
> Cc: r-help at r-project.org; R-SIG-Mixed-Models
> Subject: Re: [R] degrees of freedom and random effects in lmer
>
> I suggest this discussion be moved to the R-SIG-mixed-models mailing
> list which I am cc:ing on this reply.  Please delete the R-help
> mailing list from replies to this message.
>
> On Jan 16, 2008 11:44 AM, Feldman, Tracy <tsfeldman at noble.org> wrote:
> > Dear All,
> >
> >
> >
> > I used lmer for data with non-normally distributed error and both
> fixed
> > and random effects.  I tried to calculate a "Type III" sums of squares
> > result, by I conducting likelihood ratio tests of the full model
> against
> > a model reduced by one variable at a time (for each variable
> > separately). These tests gave appropriate degrees of freedom for each
> of
> > the two fixed effects, but when I left out one of two random effects
> > (each random effect is a categorical variable with 5 and 8 levels,
> > respectively) and tested that reduced model against the full model,
> > output showed that the test degrees of freedom = 1, which was
> incorrect.
>
> Why is that incorrect?  The degrees of freedom for a likelihood ratio
> test is usually defined as the difference in the number of parameters
> and random effects are not parameters.  They are an unobserved level
> of random variation.  The parameter associated with the random effects
> is, in the simple cases, the variance of the random effects.
>
> > Since I used an experimental design with spatial and temporal
> > "blocks"-where I repeated the same experiment several times, with a
> > different treatments in each spatial block each time (and with
> different
> > combinations of individuals in each treatment)-I am now thinking that
> I
> > should leave the random effects in the model no matter what (and only
> > test for fixed effects).  This leaves me with three related questions:
>
> > 1.      Why do Likelihood Ratio Tests of a full model against a model
> > with one less random effect report the incorrect degrees of freedom?
>
> You are more likely to get helpful responses if you avoid value
> judgements in your questions.
>
> > Are such tests treating each random variable as one combined entity?
> I
> > can provide code and data if this would help.
> >
> >
> >
> > 2.      In a publication, is it reasonable to report that I only
> tested
> > models that included random effects?  Do I need to report results of a
> > test of significance of these random effects (i.e., I am not sure how
> or
> > if I should include any information about the random effects in my
> > "ANOVA-type" tables)?
> >
> >
> >
> > 3.      If I should test for the significance of random effects, per
> se
> > (and report these), is it more appropriate to simply fit models with
> and
> > without random effects to see if the pattern of fixed effects is
> > different?  I can look at random effects using "ranef(model_name)",
> but
> > this function does not assess their significance.
> >
> >
> >
> > I am not subscribed to this list, so if possible, please reply to me
> > directly at tsfeldman at noble.org .  Thank you for your time and help.
> >
> >
> >
> > Sincerely,
> >
> >
> >
> > Tracy Feldman
> >
> >
> >         [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>