[R-sig-ME] post hocs for LMMs / GLMMs

[Adam D. I. Kramer]

Hi Kay,

The general way to do what you want is to pre-define comparisons of
interest. For example, if you thought A would be higher than B C and D, then
you would attach a contrast to your data.frame for the factor f1 that would
compare A to B C and D. If there is a contrasts() attribute on f1, then when
you fit your model (say, using lmer from the lm4 package), R will
automatically parse out and test that contrast specifically.

If you don't do this, most functions will create contrasts that compare
levels of a factor to the first level (A vs B, A vs C, A vs D in the above
case).

...but if you don't have any idea or any theory of how A B C and D differ,
I would recommend that you treat your analysis as exploratory, and then just
look at the differences without testing them and see what's there, try to
come up with a theory, and then go collect more data.  When you're just
"comparing levels" or looking at effects, there's a lot more going on than
you'd at first think--in this case, the comparisons to be made are A vs B, A
vs.  B and C, A vs.  B and D, A vs.  C and D, A vs.  B C and D, and A vs
0--8 comparisons.  The same are available for B C and D, resulting in 32
comparisons.  That's a lot!  With an interpretation alpha of .05, you may
get a couple false positives.  That is why the "reparamaterization" approach
is ill advised--it greatly inflates your likelihood of finding something by
chance alone.

So really, the best thing to do here is to encode the things you hope to
find, and test them--and if you see anything else, call it a theoretically
useful fluke. The effect is positive/negative, but you can't say it's
significant...and at that point you have to replicate it anyway so a precise
p-value isn't super useful.

--Adam

On Wed, 19 May 2010, Kay Cecil Cichini wrote:

> p.s.:
> ..."different parameterizations" may be the wrong term, as the parameters 
> actually stay the same and i only change the intercept-level.
>
> Kay Cecil Cichini schrieb:
>> hello,
>> 
>> i have several LMMs and GLMMs with 2 nominal fixed factors, f1: A,B,C,D and 
>> f2: 1,2. now i need inference on the differences of level 1 vs. 2 of f2 
>> within each level of f1, or vice versa differences of A/B, A/C, A/D, B/C, 
>> etc. within each level of f2.
>> 
>> before i try with glht(): isn't it justified to examining the model's 
>> t-tests with re-ordered levels of the nominal variables, by
>> which each of this comparisons can be yielded by the different
>> parameterizations - this seems to be the most convenient way and till now i 
>> found no one to explain to me why this may or may not be valid.
>> 
>> best regards,
>> kay
>> 
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> 
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>