[R-sig-ME] F test vs. mcmcpvalue
Hank Stevens
HStevens at MUOhio.edu
Sun Jul 6 18:54:30 CEST 2008
Hi folks,
Are there general situations in which we might expect very different
answers from F tests vs. mcmcpvalue with orthogonal contrasts (Helmert)?
I helping someone with a normal linear model with a moderate sized,
noisy data set, and I am getting very different probabilities between
F-tests and mcmcpvalue for some interactions.
I get similar F-test results whether I use lm (and ignore the random
effect of subject), lme, and lmer with an DDF approximation.
When I use mcmcpvalue, I get huge changes in P-value of a main effect
(0.6 to 0.01) when I remove its interactions. In contrast, the F-test
(using trace of the hat matrix DF's) are much more consistent when I
change the fixed effect structure.
I think mcmcpvalue is much more sensitive to overfitting the model. In
some cases, removing the interactions results in a lower AIC (with ML
fits).
In the full model, we have 28 fixed coefs (22 continuous variables or
slope interactions) and about 500 obs.
The data are VERY unbalanced.
sessionInfo()
R version 2.7.1 (2008-06-23)
i386-apple-darwin8.10.1
locale:
C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] foreign_0.8-26 Hmisc_3.4-3 lme4_0.999375-20
Matrix_0.999375-10
[5] lattice_0.17-8
loaded via a namespace (and not attached):
[1] cluster_1.11.11 grid_2.7.1 tools_2.7.1
>
Hank
More information about the R-sig-mixed-models
mailing list