[R-sig-ME] Differences in degrees of freedom between a mixed-effects model and a gls model using nlme

[Ken Beath]

I agree. I think we too often get hung up on trying to obtain the "correct"
results based on the theory. In practice that is quite often irrelevant, as
the results are much more affected by the violation of teh distributional
assumptions.

One point with these models is that assumptions of asymptotic normality are
anti-conservative, so early statisticians would have wanted to do something
to reduce this bias, and any degrees of freedom less than infinity will
give a more conservative result. These days there are better ways.

Ken

On 9 February 2015 at 21:24, Viechtbauer Wolfgang (STAT) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

> I don't want to derail this thread entirely, but it does make me wonder:
> Are people really concerned about calculating the "right" degrees of
> freedom in their applications anyway? I have pretty much stopped worrying
> about the software cleverly figuring out what the right dfs are, as I
> hardly ever deal with situations where there is a clear and correct answer
> to that question -- even in the designed experiments I see, unbalancedness
> creeps in in various ways, the most obvious one being missing data due to
> attrition (in Karl's example, there is of course a clear answer, but my
> question is more general).
>
> I am sure that the type of applications one deals with has an influence on
> this matter. If you see nicely designed experiments with balanced data,
> getting the dfs right might seem like an important concern. Or if sample
> sizes are small (as in the number of individuals and/or number of repeated
> measurements), then it may matter whether the dfs are 10 or 100 for the
> conclusions you draw from a test (which, in the end, is then based, at
> least partly, on the p-value the software throws at you). But as far as I
> am concerned, I constantly (and grudgingly, with a lot of wishful thinking)
> need to rely on the asymptotic behavior of the estimates, standard errors,
> and test statistics every which way I turn anyway. Whether the dfs are 10,
> 40.5682..., or 100 is one of my least pressing concerns. If the conclusion
> doesn't pass the interocular traumatization test, I don't have much faith
> in it anyway.
>
> I know that this has come up before, http://glmm.wikidot.com/faq
> discusses this as well, and the fact that lme4 doesn't provide p-values is,
> in essence, a statement in the same direction, but I am just curious about
> other people's opinion on this.
>
> Best,
> Wolfgang
>
> --
> Wolfgang Viechtbauer, Ph.D., Statistician
> Department of Psychiatry and Psychology
> School for Mental Health and Neuroscience
> Faculty of Health, Medicine, and Life Sciences
> Maastricht University, P.O. Box 616 (VIJV1)
> 6200 MD Maastricht, The Netherlands
> +31 (43) 388-4170 | http://www.wvbauer.com
>
> > -----Original Message-----
> > From: R-sig-mixed-models [mailto:r-sig-mixed-models-bounces at r-
> > project.org] On Behalf Of Ben Bolker
> > Sent: Monday, February 09, 2015 05:36
> > To: r-sig-mixed-models at r-project.org
> > Subject: Re: [R-sig-ME] Differences in degrees of freedom between a
> > mixed-effects model and a gls model using nlme
> >
> > Ken Beath <ken.beath at ...> writes:
> >
> > >
> > > All 3 (paired t-test, mixed effect and gls with compound symmetry) are
> > > fitting the same model, and so should give the same result. That is
> > what
> > > you see with the first example. The gls model is not getting it wrong
> > > except for the df.
> > >
> > > For the second the 3 model results should again be the same. I'm not
> > > certain why but it may be numerical. Even though the data come
> > > from a model
> > > that isn't correct for the fitting that should be irrelevant, it is the
> > > data that produce the model fit not the model that produces the data.
> > > Possibly estimates of the correlation are poor when there is little
> > > correlation, and that flows through to the mixed effects and
> > > gls results.
> > >
> > > The relationship to the unpaired t-test is probably irrelevant. Note
> > also
> > > that the default for the t.test is unequal variances whereas for a
> > mixed
> > > model it is equal variances.
> > >
> > > The df for gls is obviously in a sense a bug. Getting the df for a
> > mixed
> > > model isn't easy. Here we have a nice simple correlation structure and
> > > there is an obvious correct answer, but usually there isn't one. If the
> > > model assumed uncorrelated data then the gls df would be correct, so it
> > is
> > > necessary for the software to work out what is going on. Using
> > parametric
> > > bootstrapping to determine the underlying distribution seems a better
> > > method if accuracy is important.
> > >
> > > Ken
> > >
> >
> >   For what it's worth you can easily see what gls() is doing to
> > get its df, and confirm that it's naive, by printing nlme:::summary.gls:
> >
> >   tTable[, "p-value"] <- 2 * pt(-abs(tTable[, "t-value"]),
> >         dims$N - dims$p)
> >
> > For what it's worth, I've found that the df calculations used by
> > lme() often fail quite badly for random-slopes models ... it's often
> > really hard to guess, even for simpler designs (i.e. where there
> > really is a precise correspondence with an F distribution -- no
> > correlation
> > structures or lack of balance or crossed random effects).
> >
> > _______________________________________________
> > R-sig-mixed-models at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>



-- 

*Ken Beath*
Lecturer
Statistics Department
MACQUARIE UNIVERSITY NSW 2109, Australia

Phone: +61 (0)2 9850 8516

Building E4A, room 526
http://stat.mq.edu.au/our_staff/staff_-_alphabetical/staff/beath,_ken/

CRICOS Provider No 00002J
This message is intended for the addressee named and may...{{dropped:9}}