[R-sig-ME] Comparing models with different random effects

[Ben Bolker]

On 10-12-11 09:40 AM, Gang Chen wrote:
> Thanks a lot for the clarifications, Dr. Bolker!
> 
> Could you comment a little more about the following?
> 
>>  I was going to write more about how one would interpret the
>> relationship between Task and Subj in terms of 'effects', but I think it
>> really doesn't make sense when Task is a fixed effect.
> 
> In my case Task has only 3 levels, and that's why I didn't consider
> (1|Task/Subj) or (1|Task)+(1|Subj). Do you think (1|Task) should be
> included in the model?

  From what you've said so far it sounds to me like Task should be
treated as a fixed effect (i.e., you are interested in the actual means
for each task, or the differences among them, and not just the
variability in responses across tasks) -- so no, I would say (1|Task)
should not be in the model (that would be including it as a random
effect).  You probably do want Task in the model as a fixed effect (i.e.
something like

response~Task+(Task|Subj)+(1|Item)

> 
> Thanks,
> Gang
> 
> 
> On Fri, Dec 10, 2010 at 4:41 PM, Ben Bolker <bbolker at gmail.com> wrote:
>> On 10-12-10 10:35 AM, Yuan-Ye Zhang wrote:
>>> test the sig. of random slope, if you compare these two
>>> (1 | Subj): random intercept
>>> (Task | Subj): random intercept and random slope
>>>
>>> but can you do (Task | Subj)? I thought some levels of Subj is present in
>>> given levels of Task, so is nested.
>>>
>>
>>  (Task|Subj) means that the effect of Task varies among subjects --
>> perfectly sensible to try to estimate this if each subject answered
>> Items representing in more than one Task category.  And because
>> (Task|Subj) implicitly includes an intercept term [(Task|Subj) is
>> equivalent to (1+Task|Subj)], (1|Subj) is nested (in the sense of
>> models) within (Task|Subj).
>>
>>  I was going to write more about how one would interpret the
>> relationship between Task and Subj in terms of 'effects', but I think it
>> really doesn't make sense when Task is a fixed effect. The model
>> specification about (Task|Subj) sets up *none* of the following models:
>>
>> a. there is variation among Task:Subj combinations within a Subj ["Task
>> nested within Subj", (1|Task/Subj)], or
>> b. there is variation among Task:Subj combinations within a Task ["Subj
>> nested within Task", (1|Subj/Task)] [I may have the order of the a/b
>> notation backwards: I have yet to discover a good mnemonic], or
>> c. samples from each Task have random deviations from the overall mean
>> that are consistent across all Subj's, or vice versa ["crossed random
>> effects Subj and Task", (1|Task)+(1|Subj)]
>>
>>  instead, this sets up a model where (as stated above) the *effect* of
>> Task varies randomly (around its overall mean) across Subj's.
>>
>>  Hope that helps and that I didn't screw anything up.
>>
>>  Anyone know of a really good clear primer for this stuff (printed or
>> on the web) that includes (1) mathematical notation and (2) graphical
>> representations/example data plots for the descriptions above?
>>
>>> 2010/12/10 Gang Chen <gangchen6 at gmail.com>
>>>
>>>> Thanks for the quick help, Dr. Bates!
>>>>
>>>>>> My second question is still open: I tend to believe that (1 | Subj) is
>>>>>> nested within (Task | Subj) since the first model has one parameter
>>>>>> (variance) which can be viewed as multiple variances in the second
>>>>>> model being constrained as equal, but I would still appreciate it if
>>>>>> somebody could confirm this.
>>>>>
>>>>> If you are using "is nested within" to mean "is a submodel of" then
>>>>> the answer is yes.
>>>>
>>>> The reason I'm asking about the relationship between the two models is
>>>> whether I could use anova() to compare the two models. So, under this
>>>> context is likelihood ratio test meaningful?
>>>>
>>>> Thanks,
>>>> Gang
>>>>
>>>>
>>>>>>> 2010/12/9 Gang Chen <gangchen6 at gmail.com>
>>>>>>>>
>>>>>>>> Suppose that there are multiple task types (Task) and each task type
>>>>>>>> is represented with a few questions (Item). And all subjects (Subj)
>>>>>>>> answer the same questions (Item).
>>>>>>>>
>>>>>>>> How do I compare a model with (1 | Subj) + (1 | Item) versus one with
>>>>>>>> (1 | Subj) + (1 | Subj:Item) in lmer()? Through AIC/BIC (assuming the
>>>>>>>> fixed effect remain the same)? Would it make more sense to consider (1
>>>>>>>> | Subj) + (1 | Subj:Item) + (1 | Item)?
>>>>>>>>
>>>>>>>> Is (1 | Subj) considered as nested within (Task | Subj)?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Gang