Dear R-listers,

I have a couple of questions regrading how to assess the effects of lower
order terms when a higher order term is significant using likelihood ratio
tests. I am interested to examine how two independent variables (iv1 and
iv2) affect subjects' log-transformed reaction time (logRT). A subset of
the data can be accessed in the link below if needed:

http://www-scf.usc.edu/~hex/testset.txt

In a series of responses that Dr. Bates made in this thread:
https://stat.ethz.ch/pipermail/r-sig-mixed-models/2011q1/005399.html, he
recommended that non-significant fixed effects be removed from a model
before further llkelihood ratio tests. For example, I can first remove
*iv2* and
compare the model with both iv1 and iv2 with the model without iv2.

model.main = lmer(logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item),
Sample, REML = FALSE)
model.iv1 = lmer(logRT ~ iv1 + (1 + iv2 | subject) + (1 | item), Sample, REML
= FALSE)
anova(model.main, model.iv1)

#Data: Sample
#Models:
#model.iv1: logRT ~ iv1 + (1 + iv2 | subject) + (1 | item)
#model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item)
#           Df    AIC    BIC  logLik  Chisq Chi Df Pr(>Chisq)
#model.iv1   7 235.58 266.31 -110.79
#model.main  8 236.31 271.43 -110.15 1.2754      1     0.2588

In this case, there is no significant difference between the two models.
Based on Dr. Bates' suggestion, if I would like to further examine the
contribution of iv1, I could start from model.iv1 by comparing model.iv1
with a null model, as shown below:

model.null = lmer(logRT ~ 1 + (1 + iv2 | subject) + (1 | item), Sample, REML
= FALSE)
anova(model.iv1, model.null)

#Data: Sample
#Models:
#model.null: logRT ~ 1 + (1 + iv2 | subject) + (1 | item)
#model.iv1: logRT ~ iv1 + (1 + iv2 | subject) + (1 | item)
#           Df    AIC    BIC  logLik  Chisq Chi Df Pr(>Chisq)
#model.null  6 235.99 262.33 -112.00
#model.iv1   7 235.58 266.31 -110.79 2.4084      1     0.1207

As is shown, iv1 doesn't seem to improve to the fit of the model
significantly either.

While this procedure is clear, how does one deal with a situation where,
say, the full model has a higher order term, such as a two-way interaction,
and one would like to know not only how much the interaction contributes to
the model fit but also how much the lower order terms contribute to the
model.

model.inter = lmer(logRT ~ iv1 * iv2 + (1 + iv2 | subject) + (1 | item),
Sample, REML = FALSE)
anova(model.inter, model.main)
#Data: Sample
#Models:
#model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item)
#model.inter: logRT ~ iv1 * iv2 + (1 + iv2 | subject) + (1 | item)
#            Df    AIC    BIC  logLik  Chisq Chi Df Pr(>Chisq)
#model.main   8 236.31 271.43 -110.15
#model.inter  9 229.10 268.61 -105.55 9.2097      1   0.002407 **
#---
#Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1



(1). My first question is as follows:

For example, in the case above, the interaction is shown as significant.  I
know that when an interaction is present, main effects are often not
interpretable, so probably that makes it unnecessary to even assess main
effects? But say, if for whatever reason, I need to assess lower order
terms - for example iv2, should I compare the first pair of models below,
or the second pair of models below? It seems to me that comparing the 2nd
pair makes more sense because the significant interaction probably
shouldn't be removed from the model.

First pair:
model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item)
model.iv1:  logRT ~ iv1 + (1 + iv2 | subject) + (1 | item)

Second pair
model.main: logRT ~ iv1 + iv2 + iv1:iv2 (1 + iv2 | subject) + (1 | item)
model:      logRT ~ iv1 + iv1:iv2 + (1 + iv2 | subject) + (1 | item)

(2). My second question has to do with random effect specification. As you
can see, in all the models I showed above, I have iv2 as the random slope -
which I decided based upon comparing models that have the same fixed
effects ( iv1 * iv2) but have different nested random effect
specifications. While including iv2 makes sense for models that have iv2 as
a fixed effect (e.g., model.main, model.inter), its inclusion doesn't seem
to make sense in models where iv2 is not included as a fixed effect - for
example, the null model and model.iv1. However, it seems necessary to
include it when I compare the models below. I wonder if someone can tell me
what is the proper way of dealing with random effects in this kind of
scenario and the reasons behind.

model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item)
model.iv1:  logRT ~ iv1 + (1 + iv2 | subject) + (1 | item)


Thank you in advance!!


Best,
Xiao

	[[alternative HTML version deleted]]

