[R-sig-ME] lme approximation method for dfs

Salahadin Lotfi @@|@h@d|n@|ot|| @end|ng |rom gm@||@com
Mon May 25 23:12:24 CEST 2020 

Hi Maarten,
Sorry I kept forgetting to CC the list.
Hmm! The approximations of the approximations (the source [1] indeed
clearly mentions that). So, I figured what was the source of the error when
I tried "appx-satterthwaite" and it didn't work. I summarize that down here
for future reference if folks run into the same problem.
1) Make sure emmeans library is the most update to date (thanks Maarten to
pointing out to that). Otherwise, you might get this error below:

*Error in match.arg(mode) :   'arg' should be one of “containment”,
“satterthwaite”, “boot-satterthwaite”, “auto”*

2) If the model specification is too complex and you get *"non-positive
definite approximate variance-covariance"* for the covariance matrix, the
function* joint_tests()* with *mode =   "appx-satterthwaite" *won't work
and it throws this error:



*Error in emm_basis.lme(object, trms, xlev, grid, misc = attr(data,
"misc"),  :   Unable to estimate Satterthwaite parametersIn addition:
Warning message:In seq_len(nrow(B)) : first element used of 'length.out'
argument*


This is completely making sense (I "think" the
sattherthwaite approximation would not be possible with second-order finite
differences). Thus, this error is actually a good sign that the model is
not appropriate for the data (small data?), particularly because non-dp
approximation goes undetected by *summary() function*. Hence, run a less
complicated model.

I hope the summary is accurate.
Thanks a bunch, Maarten.
Sala

[1] https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K


*************
Salahadin (Sala) Lotfi

PhD Candidate of Cognitive Neuroscience

University of Wisconsin-Milwaukee

Anxiety Disorders Laboratory

President, Association of Clinical and Cognitive Neuroscience, UWM

On Mon, May 25, 2020 at 3:11 PM Maarten Jung <
Maarten.Jung using mailbox.tu-dresden.de> wrote:

> Hi Sala,
>
> Please keep the mailing list in cc.
>
> Looks like you don't have the latest emmeans version. In previous versions
> "appx-satterthwaite" was termed "boot-satterthwaite" (see [1]). But
> according to [1] just "satterthwaite" should also work; however, this
> misses the point that the degrees of freedom that emmeans calculates are
> *approximations to* the Satterthwaite approximation (again, see [1]).
>
> [1]
> https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K
>
> Best,
> Maarten
>
> On Mon, 25 May 2020, 21:18 Salahadin Lotfi <salahadin.lotfi using gmail.com>
> wrote:
>
>> Hi Maarten,
>> Thanks again for additional information. I am actually started to have
>> issues with *joint_tests**() *function. When I ask for  *mode =
>> "appx-satterthwaite" *, I keep receiving the below error:
>> Error in match.arg(mode) :   'arg' should be one of “containment”,
>> “satterthwaite”, “boot-satterthwaite”, “auto”
>> Precisely, when I use lmer function for fitting my model, it works with
>> no issues (I don't specify mode for that as sattherthwaite is default with
>> lmerTest()). But, when I fit the model with lme with an unstructured
>> covariance matrix (corSymm), I get the error above.
>> Would you have any input on this?
>>
>> Thanks very much,
>> Sala
>>
>> On Mon, May 25, 2020 at 3:41 AM Maarten Jung <
>> Maarten.Jung using mailbox.tu-dresden.de> wrote:
>>
>>> Hi Sala,
>>>
>>> I'm glad I could help.
>>> Please keep the mailing list in cc.
>>>
>>> Just a small addition to my last mail: If you call joint_tests()
>>> directly on your model (and not on an emmGrid object which is returned
>>> by the functions emmeans() or ref_grid()), you also have to pass the
>>> "mode" argument, otherwise you won't get the degrees of freedom you
>>> want (i.e., use joint_tests(lme_fit, mode = "appx-satterthwaite") and
>>> not just joint_tests(lme_fit)).
>>>
>>> Best,
>>> Maarten
>>>
>>>
>>>
>>> On Mon, May 25, 2020 at 7:09 AM Salahadin Lotfi
>>> <salahadin.lotfi using gmail.com> wrote:
>>> >
>>> > Maarten,
>>> >
>>> > Both suggestions were great. I used them and they are perfectly
>>> working.
>>> >
>>> > Sala
>>> >
>>> > On Sun, May 24, 2020 at 4:57 AM Maarten Jung <
>>> Maarten.Jung using mailbox.tu-dresden.de> wrote:
>>> >>
>>> >> Dear Sala,
>>> >>
>>> >> I *think* the containment method that is mentioned in the "Models
>>> >> supported by emmeans" vignette is something similar to the containment
>>> >> degrees of freedom approximation in SAS [1] but if you want to use
>>> >> this method, you should ask the author of emmeans (Russell Lenth) to
>>> >> make sure that my guess is correct.
>>> >>
>>> >> I guess your emmeans() call throws an error because you are missing
>>> >> the "specs" argument which specifies the predictor variables over
>>> >> which the estimated marginal means should be calculated. Also, the
>>> >> last argument is "sigmaAdjust" instead of "adjustSigma" but it
>>> >> defaults to TRUE anyway. This should work (if the emmeans package is
>>> >> loaded, otherwise use emmeans:emmeans()):
>>> >> emmeans(lme_fit, ~ Group * TIME, mode = "appx-satterthwaite",
>>> >> sigmaAdjust = TRUE)
>>> >>
>>> >> The function joint_tests() which constructs Type-III-ANOVA-like tables
>>> >> of all predictors and the function contrast() could be useful for your
>>> >> analysis, too. See also the nice vignettes here [2].
>>> >>
>>> >> [1]
>>> https://documentation.sas.com/?docsetId=statug&docsetTarget=statug_glimmix_details38.htm&docsetVersion=15.1&locale=en
>>> >> [2]  https://cran.r-project.org/web/packages/emmeans/vignettes/
>>> >>
>>> >> Best,
>>> >> Maarten
>>> >>
>>> >>
>>> >> On Sun, May 24, 2020 at 4:55 AM Salahadin Lotfi
>>> >> <salahadin.lotfi using gmail.com> wrote:
>>> >> >
>>> >> > Wow! Such a thorough answer, Phillip. Thanks very much.
>>> >> > I misspoken when I said the lmer would generate p values for fixed
>>> effects (yes it is indeed lmerTest package which does it).
>>> >> > Just to make sure I understand it correctly:
>>> >> > lme uses ML/REML to approximate beta weights of fixed effects,
>>> >> > it uses Wald to approximate t/z values (fixed effects), and
>>> >> > it uses "inner-outer" rule to get denominator of df (ddf) which is
>>> apparently the least optimal method to estimate ddf among other methods
>>> (etc., Satterthwaite's). Lastly, is this "inner-outer" rule the same as
>>> "containment method"?
>>> >> >
>>> >> > Maarten,
>>> >> > First of all thank you for the nice pointer. The emmeans is
>>> elegant. But, I could not get it to work. Could you point out what I am
>>> missing in the emmeans function?
>>> >> >   Here is my code:
>>> >> > ## Group and Time are categorical IVs with three levels.
>>> >> > lme_fit<- lme(DV~  factor (Group) * factor(TIME), random = ~ TIME |
>>> SubjectID, # factor(Group) * factor(TIME)
>>> >> >                   correlation = corSymm (form = ~ 1 | SubjectID),
>>> >> >                   weights = varIdent (form = ~ 1 | TIME),
>>> >> >                   data = my_data,
>>> >> >                   method= "REML",
>>> >> >                   na.action = "na.omit")
>>> >> > emmeans::emmeans(lme.fit, mode= "appx-satterthwaite", adjustSigma =
>>> TRUE) ## NOT working
>>> >> >
>>> >> >
>>> >> > I appreciate for taking time and replying.
>>> >> > Sala
>>> >> >
>>> >> > *************
>>> >> > Salahadin (Sala) Lotfi
>>> >> >
>>> >> > PhD Candidate of Cognitive Neuroscience
>>> >> >
>>> >> > University of Wisconsin-Milwaukee
>>> >> >
>>> >> > Anxiety Disorders Laboratory
>>> >> >
>>> >> > President, Association of Clinical and Cognitive Neuroscience, UWM
>>> >> >
>>> >> >
>>> >> > On Sat, May 23, 2020 at 5:01 PM Maarten Jung <
>>> Maarten.Jung using mailbox.tu-dresden.de> wrote:
>>> >> >>
>>> >> >> Dear Sala,
>>> >> >>
>>> >> >> As Phillip Alday explained, there is no implementation of the
>>> Satterthwaite approximation in the nlme package. If you want to stick with
>>> this package, the only way I know to get something similar (for lme
>>> objects) is to use functions of the emmeans package with the argument
>>> "mode" set to "appx-satterthwaite" (see [1]).
>>> >> >>
>>> >> >> [1]
>>> https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K
>>> >> >>
>>> >> >> Best,
>>> >> >> Maarten
>>> >> >>
>>> >> >>
>>> >> >> On Sat, 23 May 2020, 17:07 Phillip Alday <phillip.alday using mpi.nl>
>>> wrote:
>>> >> >>>
>>> >> >>> Hi,
>>> >> >>>
>>> >> >>> On 23/5/20 9:11 am, Salahadin Lotfi wrote:
>>> >> >>> > Dear all,
>>> >> >>> > I have a very simple question but, have been having a hard time
>>> to figure
>>> >> >>> > it out.
>>> >> >>> > I am using a mixed model with random intercept and slope using
>>> lme function
>>> >> >>> > with an unstructured covariance matrix. I know lmer uses
>>> Satterthwaite's
>>> >> >>> > approximation method to approximate dfs of fixed effects,
>>> >> >>>
>>> >> >>> This is not accurate. lme4 by default doesn't even try to figure
>>> out the
>>> >> >>> df and doesn't report p-values. The lmerTest package adds in
>>> options to
>>> >> >>> use Satterthwaite or Kenward-Roger approximations for p-values,
>>> but
>>> >> >>> depending on who you ask around here, the sentiment for those
>>> >> >>> approximations ranges from "of course" to "hmrpf, why would you
>>> bother?"
>>> >> >>> to "the heretics must be purged!".
>>> >> >>>
>>> >> >>> The GLMM FAQ (
>>> https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html)
>>> >> >>> has some info on each of these, but I'll copy and paste something
>>> >> >>> relevant that I wrote on a different mailing list:
>>> >> >>>
>>> >> >>> Treating the t-values as z-values is as reasonable as using the
>>> >> >>> t-distribution with some estimated degrees of freedom for studies
>>> with
>>> >> >>> 20-30 subjects and 10s of observations per condition per subject
>>> for two
>>> >> >>> reasons. One is that a t-distribution with dozens of degrees of
>>> freedom
>>> >> >>> is essentially a normal distribution, and so even if you could
>>> figure
>>> >> >>> out what the "right" number of degrees of freedom were, it
>>> wouldn't be
>>> >> >>> far off from the number you get from the normal distribution. The
>>> other
>>> >> >>> reason is that none of these asymptotic results are guaranteed to
>>> be
>>> >> >>> particularly great for anything other than very well behaved
>>> linear
>>> >> >>> mixed models, which is why things like parametric bootstrap are
>>> the gold
>>> >> >>> standard for figuring out coverage intervals. And for large
>>> models,
>>> >> >>> bootstrapping is about as fast as KR (because KR as implemented in
>>> >> >>> pbkrmodcomp, which lmerTest depends on, computes the inverse of a
>>> large
>>> >> >>> n x matrix).
>>> >> >>>
>>> >> >>> > but I am not sure
>>> >> >>> > what is the preferred method that lme uses. Is it Wald or
>>> Likelihood ratio?
>>> >> >>>
>>> >> >>> Wald and likelihood ratio are not degrees of freedom estimates.
>>> The
>>> >> >>> likelihood-ratio tests do have a df, which corresponds to the
>>> difference
>>> >> >>> in the number of free parameters between the models, but this not
>>> the
>>> >> >>> relevant df. (It's numerator degrees of freedom in the ANOVA
>>> framework,
>>> >> >>> while what you need are the denominator degrees of freedom.) The
>>> Wald
>>> >> >>> tests are just the things you see in the table of the fixed
>>> effects,
>>> >> >>> i.e. the tests corresponding to the t- or z-values (or more
>>> generally
>>> >> >>> the ANOVA-style tests / tests of linear hypotheses you then
>>> construct
>>> >> >>> from the fixed effects).
>>> >> >>>
>>> >> >>>
>>> >> >>> > I don't think lme offers such an option to specify an
>>> approximation method
>>> >> >>> > for dfs of fixed effects. Does it?
>>> >> >>>
>>> >> >>> The dfs in nlme are computed using the "inner-outer" rule which
>>> doesn't
>>> >> >>> work well for many types of designs common in cognitive
>>> neuroscience.
>>> >> >>> More information on this is in the GLMM FAQ, search for "Df
>>> >> >>> alternatives" on that page.
>>> >> >>>
>>> >> >>>
>>> >> >>> Hope that helps!
>>> >> >>>
>>> >> >>> Phillip
>>> >> >>>
>>> >> >>>
>>> >> >>> >
>>> >> >>> > I appreciate any response in advance.
>>> >> >>> > Sala
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > *************
>>> >> >>> > Salahadin (Sala) Lotfi
>>> >> >>> >
>>> >> >>> > PhD Candidate of Cognitive Neuroscience
>>> >> >>> >
>>> >> >>> > University of Wisconsin-Milwaukee
>>> >> >>> >
>>> >> >>> > Anxiety Disorders Laboratory
>>> >> >>> >
>>> >> >>> > President, Association of Clinical and Cognitive Neuroscience,
>>> UWM
>>> >> >>> >
>>> >> >>> >       [[alternative HTML version deleted]]
>>> >> >>> >
>>> >> >>> > _______________________________________________
>>> >> >>> > R-sig-mixed-models using r-project.org mailing list
>>> >> >>> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>> >> >>>
>>> >> >>> _______________________________________________
>>> >> >>> R-sig-mixed-models using r-project.org mailing list
>>> >> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>

	[[alternative HTML version deleted]]

More information about the R-sig-mixed-models mailing list