[R-sig-ME] F test vs. mcmcpvalue
HStevens at MUOhio.edu
Sun Jul 6 18:54:30 CEST 2008
Are there general situations in which we might expect very different
answers from F tests vs. mcmcpvalue with orthogonal contrasts (Helmert)?
I helping someone with a normal linear model with a moderate sized,
noisy data set, and I am getting very different probabilities between
F-tests and mcmcpvalue for some interactions.
I get similar F-test results whether I use lm (and ignore the random
effect of subject), lme, and lmer with an DDF approximation.
When I use mcmcpvalue, I get huge changes in P-value of a main effect
(0.6 to 0.01) when I remove its interactions. In contrast, the F-test
(using trace of the hat matrix DF's) are much more consistent when I
change the fixed effect structure.
I think mcmcpvalue is much more sensitive to overfitting the model. In
some cases, removing the interactions results in a lower AIC (with ML
In the full model, we have 28 fixed coefs (22 continuous variables or
slope interactions) and about 500 obs.
The data are VERY unbalanced.
R version 2.7.1 (2008-06-23)
attached base packages:
 stats graphics grDevices utils datasets methods base
other attached packages:
 foreign_0.8-26 Hmisc_3.4-3 lme4_0.999375-20
loaded via a namespace (and not attached):
 cluster_1.11.11 grid_2.7.1 tools_2.7.1
More information about the R-sig-mixed-models