[R] How to extract coefficients from sequential (type 1), ANOVAs using lmer and lme

[Phillip Alday]

Please reread my point #1: the tests of the (individual) coefficients in
the model summary are not the same as the ANOVA tests. There is a
certain correspondence between the two (i.e. between the coding of your
categorical variables and the type of sum of squares; and for a model
with a single predictor, F=t^2), but they are not the same in general.
The t-test in the model coefficients is simply the ratio of the estimate
to the standard error (i.e. a Wald test), and the standard errors, like
the estimates, are all calculated at the same time. So in that sense,
the t-tests are always marginal (cf. Pinheiro and Bates 2000, pp. 90-91).

I would also encourage you to focus more on your estimates (the Value
column) and less on p-values.

All that said, it seems you only care about the significance of model
terms and not the estimates, so there are two possibilities for
sequential tests:

1. Nested model comparison (likelihood-ratio test) with either the
anova() function or drop1(model,test='Chisq')

2. use the p-values from the ANOVA results

(see Pinheiro and Bates 2000, pp. 90-91, for some notes on which test is
preferred as well as the GLMM FAQ:
https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html)

But please please note that it would be misleading to say that these are
the p-values for the coefficients in your model. These issues are the
same for both mixed and 'normal' regression models.

Phillip

On 30/11/17 16:56, Akihiro Koyama wrote:
> Hi Phillip,
> 
> Thank you very much for informative comments. But I still cannot find a
> way to extract coefficients from sequential ANOVAs.
> 
> I have many data sets which all give different p-values for "sequential"
> and "marginal" options in anova(). But summary() command looks only
> provide me coefficients associated with "marginal" ANOVAs. 
> 
> 
> 
> 
> On Tuesday, November 28, 2017 6:51 AM, Phillip Alday
> <phillip.alday at mpi.nl> wrote:
> 
> 
> Be careful when mixing lme4 and lmerTest together -- lmerTest extends
> and changes the behavior of various lme4 functions.
> 
> From the help page for lme4-anova (?lme4::anova.merMod)
> 
>>      ‘anova’: returns the sequential decomposition of the contributions
>>          of fixed-effects terms or, for multiple arguments, model
>>          comparison statistics.  For objects of class ‘lmerMod’ the
>>          default behavior is to refit the models with ML if fitted
>>          with ‘REML = TRUE’, this can be controlled via the ‘refit’
>>          argument. See also ‘anova’.
> 
> So lme4-anova will give you sequential tests; note, however, that lme4
> won't calculate the denominator degrees of freedom for you and thus
> won't give p-values. See the FAQ
> (https://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-are-p_002dvalues-not-displayed-when-using-lmer_0028_0029_003f)
> 
> From the help page for lmerTest-anova (?lmerTest::anova.merModLmerTest):
>> Usage:
>>
>>      ## S4 method for signature 'merModLmerTest'
>>      anova(object, ... , ddf="Satterthwaite",
>>      type=3)
>>     
>> Arguments:
>>
> ...
>>    type: type of hypothesis to be tested. Could be type=3 or type=2 or
>>          type = 1 (The definition comes from SAS theory)
> 
> 
> So lmerTest-anova by default gives you Type III ('marginal', although
> Type II is what actually gives you tests that respect the Principle of
> Marginality; see John Fox's Applied Regression Analysis (book) or
> Venables' "Exegeses on Linear Models"
> (https://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf) for more information
> on that. Type I tests are the sequential tests, so with anova(model,
> type=1), you will get the sequential tests you want. lmerTest will
> approximate the denominator degrees of freedom for you (using
> Satterthwaite method by default, or the more computationally intensive
> Kenward-Roger method), so you'll get p-values if that's what you want.
> 
> Finally, it's important to note two things:
> 
> 1. The "type"-argument for nlme::summary doesn't actually do anything
> (see ?nlme::summary.lme). It's just passed onto the 'print' method,
> where it's silently ignored. The 'type' of sum of squares is an
> ANOVA-thing; the closest correspondence in terms of model coefficients
> is the coding of your categorical contrasts. See the literature
> mentioned above for more details as well as Dale Barr's discussion on
> simple vs. main effects in regression models
> (http://talklab.psy.gla.ac.uk/tvw/catpred/).
> 
> (?nlme::anova.lme does have indeed have a 'type' argument.)
> 
> 2. It is possible for the sequential tests and the marginal tests to
> yield the same results. Again, see the above literature. You have no
> interactions in your model and continuous (i.e. not-categorical)
> predictors, so if they're orthogonal, then the sequential and marginal
> tests will be numerically the same, even if they test different
> hypotheses. (See section 5.2, starting on page 14; the sequential tests
> are the "eliminating" tests, while the marginal tests are the "ignoring"
> tests in that explanation.)
> 
> Best,
> Phillip
> 
> 
> On 28/11/17 12:00, r-help-request at r-project.org
> <mailto:r-help-request at r-project.org> wrote:
>> I wantto run sequential ANOVAs (i.e. type I sums of squares), and
> trying to getresults including ANOVA tables and associated coefficients
> for predictive variables(I am using the R 3.4.2 version). I think ANOVA
> tables look right, but believecoefficients are wrong. Specifically, it
> looks like that the coefficients arefrom ANOVA with ?marginal? (type III
> sums of squares). I have tried both lme (nlmepackage) and lmer (lme4 +
> lmerTEST packages). Examples of the results arebelow:
>>
> 
> 
> <snip>
> 
> 
>> Ibelieve the results from summary() are for ?marginal? instead of
> ?sequential?ANOVA because the p-value (i.e., 0.237 for narea) in summary
> are identical tothose in tables from ?marginal?. I also used lmer in the
> lme4 pacakge to findthe same results (summary() results look like from
> ?marginal?).
>>
>>
>> Cananybody tell me how to get coefficients for ?sequential? ANOVAs?
> Thank you.
>>
> 
> 
>