[R-meta] AICc or variance components, which one matters more?

[Viechtbauer, Wolfgang (SP)]

In g3, all variance components are identifiable.

What have you found out about g4?

>-----Original Message-----
>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>Sent: Monday, 15 November, 2021 18:28
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: [R-meta] AICc or variance components, which one matters more?
>
>Hi Wolfgang,
>
>Could you possibly elaborate a bit on "in this case, yes"?
>
>This allows me to better justify g3 or g4 models to my co-authors.
>
>Thanks again,
>Luke
>
>On Mon, Nov 15, 2021, 11:19 AM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>-----Original Message-----
>>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>>Sent: Saturday, 13 November, 2021 21:10
>>To: Viechtbauer, Wolfgang (SP)
>>Cc: R meta
>>Subject: Re: [R-meta] AICc or variance components, which one matters more?
>>
>>Interesting! To make sure I'm following you, your suggested g3 model
>>both considers 'study' to be nested in the 'lab', and at the same time
>>it considers 'study' to have its own independent crossed effect. Can
>>we consider the same variable (e.g., study) to be both nested and
>>crossed at the same time?
>
>In this case, yes.
>
>>If so, I can then suggest the following model as well:
>>
>>(g4=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | lab), data = dd))
>
>I suggest you profile the variance components from that model and draw
>appropriate conclusions.
>
>>Doesn't this denote that one is uncertain about whether to take a
>>variable as nested or crossed or there are other justifications?
>>
>>Thank you,
>>Luke
>>
>>On Sat, Nov 13, 2021 at 1:11 PM Viechtbauer, Wolfgang (SP)
>><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>>
>>> >-----Original Message-----
>>> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>>> >Sent: Saturday, 13 November, 2021 18:44
>>> >To: Viechtbauer, Wolfgang (SP)
>>> >Cc: Philippe Tadger; R meta
>>> >Subject: Re: [R-meta] AICc or variance components, which one matters more?
>>> >
>>> >Hi Wolfgang,
>>> >
>>> >I'm fully with you, however, in my data only once 2 labs (labs 1 and
>>> >2) have collaborated on study 2. Specifically, part of study 2 has
>>> >been carried out by lab 1 (one row) and part of it by lab 2 (one row).
>>> >Except in this case, no such between-lab collaborations have ever
>>> >occurred in the data.
>>> >
>>> >If such a between-lab collaboration didn't exist, I could directly go
>>> >for g1 (hierarchical model). But with this collaboration, there is
>>> >just a tiny possibility for g2 (crossed model) as well.
>>> >
>>> >So, do you think AICc should be the basis of the comparison between g1
>>> >vs. g2 or the dominant data structure (ignoring the one exception)?
>>>
>>> Using information criteria *could* be the basis. But I might be inclined to
>>just ignore the issue you describe above though if this only affects one study.
>>>
>>> Just as a note: It's not necessarily an either-or choice. This model is also
>>possible:
>>>
>>> (g3=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | study), data = dd))
>>>
>>> and profile(g3) suggests that all variance components are identifiable -
>>although of course this is quite overfitted with so little data.
>>>
>>> >(g1=rma.mv(yi, vi, random = ~1|lab/study, data = dd))
>>> >(g2=rma.mv(yi, vi, random = list(~1|lab, ~1|study), data = dd))
>>> >
>>> >Thanks,
>>> >Luke