[R-sig-ME] lme approximation method for dfs

[Maarten Jung]

Hi Sala,

Please keep the mailing list in cc.

Looks like you don't have the latest emmeans version. In previous versions
"appx-satterthwaite" was termed "boot-satterthwaite" (see [1]). But
according to [1] just "satterthwaite" should also work; however, this
misses the point that the degrees of freedom that emmeans calculates are
*approximations to* the Satterthwaite approximation (again, see [1]).

[1] https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K

Best,
Maarten

On Mon, 25 May 2020, 21:18 Salahadin Lotfi <salahadin.lotfi using gmail.com>
wrote:

> Hi Maarten,
> Thanks again for additional information. I am actually started to have
> issues with *joint_tests**() *function. When I ask for  *mode =
> "appx-satterthwaite" *, I keep receiving the below error:
> Error in match.arg(mode) :   'arg' should be one of “containment”,
> “satterthwaite”, “boot-satterthwaite”, “auto”
> Precisely, when I use lmer function for fitting my model, it works with no
> issues (I don't specify mode for that as sattherthwaite is default with
> lmerTest()). But, when I fit the model with lme with an unstructured
> covariance matrix (corSymm), I get the error above.
> Would you have any input on this?
>
> Thanks very much,
> Sala
>
> On Mon, May 25, 2020 at 3:41 AM Maarten Jung <
> Maarten.Jung using mailbox.tu-dresden.de> wrote:
>
>> Hi Sala,
>>
>> I'm glad I could help.
>> Please keep the mailing list in cc.
>>
>> Just a small addition to my last mail: If you call joint_tests()
>> directly on your model (and not on an emmGrid object which is returned
>> by the functions emmeans() or ref_grid()), you also have to pass the
>> "mode" argument, otherwise you won't get the degrees of freedom you
>> want (i.e., use joint_tests(lme_fit, mode = "appx-satterthwaite") and
>> not just joint_tests(lme_fit)).
>>
>> Best,
>> Maarten
>>
>>
>>
>> On Mon, May 25, 2020 at 7:09 AM Salahadin Lotfi
>> <salahadin.lotfi using gmail.com> wrote:
>> >
>> > Maarten,
>> >
>> > Both suggestions were great. I used them and they are perfectly working.
>> >
>> > Sala
>> >
>> > On Sun, May 24, 2020 at 4:57 AM Maarten Jung <
>> Maarten.Jung using mailbox.tu-dresden.de> wrote:
>> >>
>> >> Dear Sala,
>> >>
>> >> I *think* the containment method that is mentioned in the "Models
>> >> supported by emmeans" vignette is something similar to the containment
>> >> degrees of freedom approximation in SAS [1] but if you want to use
>> >> this method, you should ask the author of emmeans (Russell Lenth) to
>> >> make sure that my guess is correct.
>> >>
>> >> I guess your emmeans() call throws an error because you are missing
>> >> the "specs" argument which specifies the predictor variables over
>> >> which the estimated marginal means should be calculated. Also, the
>> >> last argument is "sigmaAdjust" instead of "adjustSigma" but it
>> >> defaults to TRUE anyway. This should work (if the emmeans package is
>> >> loaded, otherwise use emmeans:emmeans()):
>> >> emmeans(lme_fit, ~ Group * TIME, mode = "appx-satterthwaite",
>> >> sigmaAdjust = TRUE)
>> >>
>> >> The function joint_tests() which constructs Type-III-ANOVA-like tables
>> >> of all predictors and the function contrast() could be useful for your
>> >> analysis, too. See also the nice vignettes here [2].
>> >>
>> >> [1]
>> https://documentation.sas.com/?docsetId=statug&docsetTarget=statug_glimmix_details38.htm&docsetVersion=15.1&locale=en
>> >> [2]  https://cran.r-project.org/web/packages/emmeans/vignettes/
>> >>
>> >> Best,
>> >> Maarten
>> >>
>> >>
>> >> On Sun, May 24, 2020 at 4:55 AM Salahadin Lotfi
>> >> <salahadin.lotfi using gmail.com> wrote:
>> >> >
>> >> > Wow! Such a thorough answer, Phillip. Thanks very much.
>> >> > I misspoken when I said the lmer would generate p values for fixed
>> effects (yes it is indeed lmerTest package which does it).
>> >> > Just to make sure I understand it correctly:
>> >> > lme uses ML/REML to approximate beta weights of fixed effects,
>> >> > it uses Wald to approximate t/z values (fixed effects), and
>> >> > it uses "inner-outer" rule to get denominator of df (ddf) which is
>> apparently the least optimal method to estimate ddf among other methods
>> (etc., Satterthwaite's). Lastly, is this "inner-outer" rule the same as
>> "containment method"?
>> >> >
>> >> > Maarten,
>> >> > First of all thank you for the nice pointer. The emmeans is elegant.
>> But, I could not get it to work. Could you point out what I am missing in
>> the emmeans function?
>> >> >   Here is my code:
>> >> > ## Group and Time are categorical IVs with three levels.
>> >> > lme_fit<- lme(DV~  factor (Group) * factor(TIME), random = ~ TIME |
>> SubjectID, # factor(Group) * factor(TIME)
>> >> >                   correlation = corSymm (form = ~ 1 | SubjectID),
>> >> >                   weights = varIdent (form = ~ 1 | TIME),
>> >> >                   data = my_data,
>> >> >                   method= "REML",
>> >> >                   na.action = "na.omit")
>> >> > emmeans::emmeans(lme.fit, mode= "appx-satterthwaite", adjustSigma =
>> TRUE) ## NOT working
>> >> >
>> >> >
>> >> > I appreciate for taking time and replying.
>> >> > Sala
>> >> >
>> >> > *************
>> >> > Salahadin (Sala) Lotfi
>> >> >
>> >> > PhD Candidate of Cognitive Neuroscience
>> >> >
>> >> > University of Wisconsin-Milwaukee
>> >> >
>> >> > Anxiety Disorders Laboratory
>> >> >
>> >> > President, Association of Clinical and Cognitive Neuroscience, UWM
>> >> >
>> >> >
>> >> > On Sat, May 23, 2020 at 5:01 PM Maarten Jung <
>> Maarten.Jung using mailbox.tu-dresden.de> wrote:
>> >> >>
>> >> >> Dear Sala,
>> >> >>
>> >> >> As Phillip Alday explained, there is no implementation of the
>> Satterthwaite approximation in the nlme package. If you want to stick with
>> this package, the only way I know to get something similar (for lme
>> objects) is to use functions of the emmeans package with the argument
>> "mode" set to "appx-satterthwaite" (see [1]).
>> >> >>
>> >> >> [1]
>> https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K
>> >> >>
>> >> >> Best,
>> >> >> Maarten
>> >> >>
>> >> >>
>> >> >> On Sat, 23 May 2020, 17:07 Phillip Alday <phillip.alday using mpi.nl>
>> wrote:
>> >> >>>
>> >> >>> Hi,
>> >> >>>
>> >> >>> On 23/5/20 9:11 am, Salahadin Lotfi wrote:
>> >> >>> > Dear all,
>> >> >>> > I have a very simple question but, have been having a hard time
>> to figure
>> >> >>> > it out.
>> >> >>> > I am using a mixed model with random intercept and slope using
>> lme function
>> >> >>> > with an unstructured covariance matrix. I know lmer uses
>> Satterthwaite's
>> >> >>> > approximation method to approximate dfs of fixed effects,
>> >> >>>
>> >> >>> This is not accurate. lme4 by default doesn't even try to figure
>> out the
>> >> >>> df and doesn't report p-values. The lmerTest package adds in
>> options to
>> >> >>> use Satterthwaite or Kenward-Roger approximations for p-values, but
>> >> >>> depending on who you ask around here, the sentiment for those
>> >> >>> approximations ranges from "of course" to "hmrpf, why would you
>> bother?"
>> >> >>> to "the heretics must be purged!".
>> >> >>>
>> >> >>> The GLMM FAQ (
>> https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html)
>> >> >>> has some info on each of these, but I'll copy and paste something
>> >> >>> relevant that I wrote on a different mailing list:
>> >> >>>
>> >> >>> Treating the t-values as z-values is as reasonable as using the
>> >> >>> t-distribution with some estimated degrees of freedom for studies
>> with
>> >> >>> 20-30 subjects and 10s of observations per condition per subject
>> for two
>> >> >>> reasons. One is that a t-distribution with dozens of degrees of
>> freedom
>> >> >>> is essentially a normal distribution, and so even if you could
>> figure
>> >> >>> out what the "right" number of degrees of freedom were, it
>> wouldn't be
>> >> >>> far off from the number you get from the normal distribution. The
>> other
>> >> >>> reason is that none of these asymptotic results are guaranteed to
>> be
>> >> >>> particularly great for anything other than very well behaved linear
>> >> >>> mixed models, which is why things like parametric bootstrap are
>> the gold
>> >> >>> standard for figuring out coverage intervals. And for large models,
>> >> >>> bootstrapping is about as fast as KR (because KR as implemented in
>> >> >>> pbkrmodcomp, which lmerTest depends on, computes the inverse of a
>> large
>> >> >>> n x matrix).
>> >> >>>
>> >> >>> > but I am not sure
>> >> >>> > what is the preferred method that lme uses. Is it Wald or
>> Likelihood ratio?
>> >> >>>
>> >> >>> Wald and likelihood ratio are not degrees of freedom estimates. The
>> >> >>> likelihood-ratio tests do have a df, which corresponds to the
>> difference
>> >> >>> in the number of free parameters between the models, but this not
>> the
>> >> >>> relevant df. (It's numerator degrees of freedom in the ANOVA
>> framework,
>> >> >>> while what you need are the denominator degrees of freedom.) The
>> Wald
>> >> >>> tests are just the things you see in the table of the fixed
>> effects,
>> >> >>> i.e. the tests corresponding to the t- or z-values (or more
>> generally
>> >> >>> the ANOVA-style tests / tests of linear hypotheses you then
>> construct
>> >> >>> from the fixed effects).
>> >> >>>
>> >> >>>
>> >> >>> > I don't think lme offers such an option to specify an
>> approximation method
>> >> >>> > for dfs of fixed effects. Does it?
>> >> >>>
>> >> >>> The dfs in nlme are computed using the "inner-outer" rule which
>> doesn't
>> >> >>> work well for many types of designs common in cognitive
>> neuroscience.
>> >> >>> More information on this is in the GLMM FAQ, search for "Df
>> >> >>> alternatives" on that page.
>> >> >>>
>> >> >>>
>> >> >>> Hope that helps!
>> >> >>>
>> >> >>> Phillip
>> >> >>>
>> >> >>>
>> >> >>> >
>> >> >>> > I appreciate any response in advance.
>> >> >>> > Sala
>> >> >>> >
>> >> >>> >
>> >> >>> > *************
>> >> >>> > Salahadin (Sala) Lotfi
>> >> >>> >
>> >> >>> > PhD Candidate of Cognitive Neuroscience
>> >> >>> >
>> >> >>> > University of Wisconsin-Milwaukee
>> >> >>> >
>> >> >>> > Anxiety Disorders Laboratory
>> >> >>> >
>> >> >>> > President, Association of Clinical and Cognitive Neuroscience,
>> UWM
>> >> >>> >
>> >> >>> >       [[alternative HTML version deleted]]
>> >> >>> >
>> >> >>> > _______________________________________________
>> >> >>> > R-sig-mixed-models using r-project.org mailing list
>> >> >>> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> R-sig-mixed-models using r-project.org mailing list
>> >> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>

	[[alternative HTML version deleted]]