[R-meta] AICc or variance components, which one matters more?

[Viechtbauer, Wolfgang (SP)]

>-----Original Message-----
>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>Sent: Saturday, 13 November, 2021 21:10
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: [R-meta] AICc or variance components, which one matters more?
>
>Interesting! To make sure I'm following you, your suggested g3 model
>both considers 'study' to be nested in the 'lab', and at the same time
>it considers 'study' to have its own independent crossed effect. Can
>we consider the same variable (e.g., study) to be both nested and
>crossed at the same time?

In this case, yes.

>If so, I can then suggest the following model as well:
>
>(g4=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | lab), data = dd))

I suggest you profile the variance components from that model and draw appropriate conclusions.

>Doesn't this denote that one is uncertain about whether to take a
>variable as nested or crossed or there are other justifications?
>
>Thank you,
>Luke
>
>On Sat, Nov 13, 2021 at 1:11 PM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>
>> >-----Original Message-----
>> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>> >Sent: Saturday, 13 November, 2021 18:44
>> >To: Viechtbauer, Wolfgang (SP)
>> >Cc: Philippe Tadger; R meta
>> >Subject: Re: [R-meta] AICc or variance components, which one matters more?
>> >
>> >Hi Wolfgang,
>> >
>> >I'm fully with you, however, in my data only once 2 labs (labs 1 and
>> >2) have collaborated on study 2. Specifically, part of study 2 has
>> >been carried out by lab 1 (one row) and part of it by lab 2 (one row).
>> >Except in this case, no such between-lab collaborations have ever
>> >occurred in the data.
>> >
>> >If such a between-lab collaboration didn't exist, I could directly go
>> >for g1 (hierarchical model). But with this collaboration, there is
>> >just a tiny possibility for g2 (crossed model) as well.
>> >
>> >So, do you think AICc should be the basis of the comparison between g1
>> >vs. g2 or the dominant data structure (ignoring the one exception)?
>>
>> Using information criteria *could* be the basis. But I might be inclined to
>just ignore the issue you describe above though if this only affects one study.
>>
>> Just as a note: It's not necessarily an either-or choice. This model is also
>possible:
>>
>> (g3=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | study), data = dd))
>>
>> and profile(g3) suggests that all variance components are identifiable -
>although of course this is quite overfitted with so little data.
>>
>> >(g1=rma.mv(yi, vi, random = ~1|lab/study, data = dd))
>> >(g2=rma.mv(yi, vi, random = list(~1|lab, ~1|study), data = dd))
>> >
>> >Thanks,
>> >Luke