[R-sig-ME] How to obtain Type 2 like p-values for effects?

[Henrik Singmann]

Dear all,

I am currently trying to add the possibility to obtain Type 2 like p-values for all effects in a mixed model to function mixed() in package afex (using pbkrtest::KRmodcomp see: https://stat.ethz.ch/pipermail/r-sig-mixed-models/2012q3/018946.html) but am unsure on how to implement this correctly.

I came up with two possible solutions which both seem to have problems.

(1) Fit one full model and contrast a model for each effect with this full model. Each submodel is created by subtracting the effect of interest and all higher order effects from the model (e.g., when interested in a main effect, the submodel contains all main effects but the one of interest and no interactions).

(2) Fit a full model for each order of effects (i.e., one full model for all main effects, one full model for all two-way interactions, ...) and contrast a model for each effect with the full model of the corresponding order (e.g., when interested in a main effect, the submodel contains all main effects but the one of interest and no interactions and the full model contains all main effects and no interactions).

Both solutions seem to have problems.
The first solution seems rather dubious, why would you want to compare two models to test for a single effect when in fact more than just this effect is missing, namely all the higher order effects. Consequently, this solution does produces small p-values and many significant effects. For example, comparing a full model with many interactions with a model for the main effects in which only those are present almost inevitably needs to be significant.
The second solution seems problematic from another perspective, as it simply ignores the higher order effects. This contrasts with classical ANOVA in which even the lower order effects are tested against the MSE of the full model. Consequently, using the second approach the lower order effects of a model with interactions are identical to a model fitted without those interactions.

Any ideas would be really appreciated.

(Sidenote: I obtain type 3 tests by contrasting the full model with a model in which only the relevant effect is missing, ignoring the order of the effects)

Cheers,
Henrik

-- 
Dipl. Psych. Henrik Singmann
PhD Student
Albert-Ludwigs-Universität Freiburg, Germany
http://www.psychologie.uni-freiburg.de/Members/singmann