[R-sig-ME] Use LRT to assess the effect of lower order terms in the presence of a significant higher order term (e.g., interaction)

[David Duffy]

On Sat, 19 May 2012, Xiao He wrote:

> I have a couple of questions regrading how to assess the effects of lower
> order terms when a higher order term is significant using likelihood ratio
> tests. I am interested to examine how two independent variables (iv1 and
> iv2) affect subjects' log-transformed reaction time (logRT).
>
> In a series of responses that Dr. Bates made in this thread:
> https://stat.ethz.ch/pipermail/r-sig-mixed-models/2011q1/005399.html, he
> recommended that non-significant fixed effects be removed from a model
> before further llkelihood ratio tests.

[snip]

> #Models:
> #model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item)
> #model.inter: logRT ~ iv1 * iv2 + (1 + iv2 | subject) + (1 | item)
> #            Df    AIC    BIC  logLik  Chisq Chi Df Pr(>Chisq)
> #model.main   8 236.31 271.43 -110.15
> #model.inter  9 229.10 268.61 -105.55 9.2097      1   0.002407 **
>
> (1). My first question is as follows:
>
> For example, in the case above, the interaction is shown as significant.  I
> know that when an interaction is present, main effects are often not
> interpretable, so probably that makes it unnecessary to even assess main
> effects? But say, if for whatever reason, I need to assess lower order
> terms - for example iv2, should I compare the first pair of models below,
> or the second pair of models below? It seems to me that comparing the 2nd
> pair makes more sense because the significant interaction probably
> shouldn't be removed from the model.
>
> First pair:
> model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item)
> model.iv1:  logRT ~ iv1 + (1 + iv2 | subject) + (1 | item)
>
> Second pair
> model.main: logRT ~ iv1 + iv2 + iv1:iv2 (1 + iv2 | subject) + (1 | item)
> model:      logRT ~ iv1 + iv1:iv2 + (1 + iv2 | subject) + (1 | item)

Did you actually try the second one?

> (2). My second question has to do with random effect specification. As you
> can see, in all the models I showed above, I have iv2 as the random slope -
> which I decided based upon comparing models that have the same fixed
> effects ( iv1 * iv2) but have different nested random effect
> specifications. While including iv2 makes sense for models that have iv2 as
> a fixed effect (e.g., model.main, model.inter), its inclusion doesn't seem
> to make sense in models where iv2 is not included as a fixed effect - for
> example, the null model and model.iv1. However, it seems necessary to
> include it when I compare the models below. I wonder if someone can tell me
> what is the proper way of dealing with random effects in this kind of
> scenario and the reasons behind.
>
> model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item)
> model.iv1:  logRT ~ iv1 + (1 + iv2 | subject) + (1 | item)

I think all this requires you to consult locally about your exact problem 
domain.  Would you expact iv1 and iv2 to interact?  Would you expect a 
subject specific slope on iv2?  If iv2, why not iv1?  If you choose a 
different transformation of RT, some of these complications might 
disappear, so that the interpretation will be cleaner.

If iv2 is a well known important covariate of RT from the literature *or* 
subject specific slopes are needed, iv2 should be in as a main effect. 
The next questions are whether you also need iv1 in the random effects 
part of the model (given you find an iv1:iv2 interaction), and whether you 
have enough data to support estimating all these effects.

Cheers, David Duffy,