[R-sig-ME] Replicating type III anova tests for glmer/GLMM

Fox, John jfox at mcmaster.ca
Wed Feb 24 17:08:42 CET 2016

Dear Emmanuel,

The questions you raise are sufficiently complicated that it's difficult to address them adequately in an email. My Applied Regression and Generalized Linear Models text, for example, takes about 15 pages to explain the relationships among regressor codings, hypotheses, and tests in 2-way ANOVA, working with the full-rank parametrization of the model, and it's possible (as Russell Lenth indicated) to work things out even more generally. 

I'll try to answer briefly, however.

> -----Original Message-----
> From: Emmanuel Curis [mailto:emmanuel.curis at parisdescartes.fr]
> Sent: February 24, 2016 4:54 AM
> To: Fox, John <jfox at mcmaster.ca>
> Cc: Francesco Romano <francescobryanromano at gmail.com>; r-sig-mixed-
> models at r-project.org
> Subject: Re: [R-sig-ME] Replicating type III anova tests for glmer/GLMM
> Dear Pr Fox,
> Thanks for taking time for this discussion. I think I made a few shortcuts that
> are wrong, and I still have some not understood issues about the kind of
> tests even in the simplest case of linear models...
> First, I think I mixed contrast and quadratic forms expectations in my answer,
> I apologize for that; what I had in mind when answering Francesco was in fact

No need to apologize. I don't think that these are simple ideas.

> the expectation of the quadratic form, and I too quickly deduced that there
> was an equivalent linear combination of the parameters as its « square root
> », but this was obviously wrong since the L matrix in a Lt W L quadratic form
> does not have to be a column matrix. Am I wrong thinking that typically in
> such tests, the L matrix is precisely a multi-column matrix (hence also several
> degrees of freedom associated), and that several contrasts are tested
> simultaneously?

Thinking in terms of the full-rank parametrization, as used in R, each type-III hypothesis is that several coefficients are simultaneously 0, which can be simply formulated as a linear hypothesis assuming an appropriate coding of the regressors for a factor. Type-II hypotheses can also be formulated as linear hypotheses, but doing so is more complicated. The Anova() function uses a kind of projection, in effect defining a type-II test as the most powerful test of a conditional hypothesis such as no A main effect given that the A:B interaction is absent in the model y ~ A*B. This works both for linear models, where (unless there is a complication like missing cells), the resulting test corresponds to the test produced by comparing the models y ~ A and y ~ A + B, using Y ~ A*B for the estimate of error variance (i.e., the denominator MS), and more generally for models with linear predictors, where it's in general possible to formulate the (Wald) tests in terms of the coefficient estimates and their covariance matrix.

> I precise that I call « contrast » a linear combination of the model parameters
> with the constraint that the coefficients of this combination sum to 0 ─ this is
> the definition in French (« contraste »), but I may use it wrongly in English?

I'd define a "contrast" as the weights associated with the levels of a factor for formulating a hypothesis, where the weights traditionally are constrained to sum to 0, and to differentiate this from a column of the model matrix, which I'd more generally term a "regressor." Often, a traditional set of contrasts for a factor, one less than the number of levels, are defined not only to sum to 0  but also to be orthogonal in the basis of the design. The usage in R is more general, where "contrasts" mean the set of regressors used to represent a factor. Thus, contr.sum() generates regressors that satisfy the traditional definition of contrasts, as do contr.poly() and contr.helmert(), but the default contr.treatment() generates 0/1 dummy-coded regressors that don't satisfy the traditional definition of contrasts.

> Second, I may have wrongly understood the definitions of the various tests,
> and especially how they generalize from linear model to GLM/GLMM...
> I thought type I was by taking the squared distance of the successive
> orthogonal projections on the subspaces generated by the various terms, in
> the order given in the model; type II, by ensuring that the term tested was
> the last amongst terms of same order, after terms of lower order but before
> terms of higher order; type III, by projecting on the subspace after removal
> of the basis vectors for the term tested ─ hence its strong dependency on
> the coding scheme, and the « drop1 » trick to get them.
> Is this definition correct? Does it generalize to other kind models, or is
> another definition required? Is it unambiguous? The SAS doc itself suggests
> that various procedures call "type II" different kind of things

Yes, if I've followed this correctly, it's correct, and it explains why it's possible to formulate the different types of tests in linear models independently of the contrasts (regressors) used to code the factors -- because fundamentally what's important is the subspace spanned by the regressors in each model, which is independent of coding. This approach, however, doesn't generalize easily beyond linear models fit by least squares. The approach taken in Anova() corresponds to this approach in linear models fit by least squares as long as the models remain full-rank and for type-III tests as long as the contrasts are properly formulated, and generalizes to other models with linear predictors.

> However, I cannot see clearly which hypothesis is indeed tested in each case,
> especially in terms of cell means or marginal means (and, when I really need
> it, I start from them and select the contrasts I need).  Is there any
> package/software that allows to print the hypotheses testeds in terms of
> means starting from the model formula?
> Or is there any good reference that makes the link between the two?
> For instance, a demonstration that the comparison of marginal means «
> always » leads to a type XXX sum of square?

This is where a complete explanation gets too lengthy for an email, but a shorthand formulation, e.g., for the model y ~ A*B, is that type-I tests correspond to the hypotheses A|(B = 0, AB = 0), B | AB = 0, AB = 0; type-II tests to A | AB = 0, B | AB = 0, AB = 0; and type-III tests to A = 0, B = 0, AB = 0. Here, e.g., | AB = 0 means assuming no AB interactions, so, e.g., the hypothesis A | AB = 0 means no A main effects assuming no AB interactions. A hypothesis like A = 0 is indeed formulated in terms of marginal means, understood as cell means for A averaging over the levels of B (not level means of A ignoring B).

I realize that this is far from a complete explanation.


> Best regards,
> On Tue, Feb 23, 2016 at 05:17:29PM +0000, Fox, John wrote:
> « Dear Emmanuel,
> «
> « First, the relevant linear hypothesis is for several coefficients
> simultaneously -- for example, all 3 coefficients for the contrasts
> representing a 4-level factor -- not for a single contrast. Although it's true
> that any linear combination of parameters that are 0 is 0, the converse isn't
> true. Second, for a GLMM, we really should be talking about type-III tests not
> type-III sums of squares.
> «
> « Type-III tests are dependent on coding in the full-rank parametrization of
> linear (and similar) models used in R, to make the tests correspond to
> reasonable hypotheses. The invariance of type-II tests with respect to coding
> is attractive, but shouldn't distract from the fundamental issues, which are
> the hypotheses that are tested and the power of the tests.
> «
> « Best,
> «  John
> --
>                                 Emmanuel CURIS
>                                 emmanuel.curis at parisdescartes.fr
> Page WWW: http://emmanuel.curis.online.fr/index.html

More information about the R-sig-mixed-models mailing list