[R-meta] AICc or variance components, which one matters more?

[Luke Martinez]

In g4, also, all variance components are identifiable (and larger in
magnitude).

But what's the thinking behind either g3 or g4 where the same grouping
variable is both nested and crossed?



On Mon, Nov 15, 2021, 11:33 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> In g3, all variance components are identifiable.
>
> What have you found out about g4?
>
> >-----Original Message-----
> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >Sent: Monday, 15 November, 2021 18:28
> >To: Viechtbauer, Wolfgang (SP)
> >Cc: R meta
> >Subject: Re: [R-meta] AICc or variance components, which one matters more?
> >
> >Hi Wolfgang,
> >
> >Could you possibly elaborate a bit on "in this case, yes"?
> >
> >This allows me to better justify g3 or g4 models to my co-authors.
> >
> >Thanks again,
> >Luke
> >
> >On Mon, Nov 15, 2021, 11:19 AM Viechtbauer, Wolfgang (SP)
> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >>-----Original Message-----
> >>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >>Sent: Saturday, 13 November, 2021 21:10
> >>To: Viechtbauer, Wolfgang (SP)
> >>Cc: R meta
> >>Subject: Re: [R-meta] AICc or variance components, which one matters
> more?
> >>
> >>Interesting! To make sure I'm following you, your suggested g3 model
> >>both considers 'study' to be nested in the 'lab', and at the same time
> >>it considers 'study' to have its own independent crossed effect. Can
> >>we consider the same variable (e.g., study) to be both nested and
> >>crossed at the same time?
> >
> >In this case, yes.
> >
> >>If so, I can then suggest the following model as well:
> >>
> >>(g4=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | lab), data = dd))
> >
> >I suggest you profile the variance components from that model and draw
> >appropriate conclusions.
> >
> >>Doesn't this denote that one is uncertain about whether to take a
> >>variable as nested or crossed or there are other justifications?
> >>
> >>Thank you,
> >>Luke
> >>
> >>On Sat, Nov 13, 2021 at 1:11 PM Viechtbauer, Wolfgang (SP)
> >><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >>>
> >>> >-----Original Message-----
> >>> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >>> >Sent: Saturday, 13 November, 2021 18:44
> >>> >To: Viechtbauer, Wolfgang (SP)
> >>> >Cc: Philippe Tadger; R meta
> >>> >Subject: Re: [R-meta] AICc or variance components, which one matters
> more?
> >>> >
> >>> >Hi Wolfgang,
> >>> >
> >>> >I'm fully with you, however, in my data only once 2 labs (labs 1 and
> >>> >2) have collaborated on study 2. Specifically, part of study 2 has
> >>> >been carried out by lab 1 (one row) and part of it by lab 2 (one row).
> >>> >Except in this case, no such between-lab collaborations have ever
> >>> >occurred in the data.
> >>> >
> >>> >If such a between-lab collaboration didn't exist, I could directly go
> >>> >for g1 (hierarchical model). But with this collaboration, there is
> >>> >just a tiny possibility for g2 (crossed model) as well.
> >>> >
> >>> >So, do you think AICc should be the basis of the comparison between g1
> >>> >vs. g2 or the dominant data structure (ignoring the one exception)?
> >>>
> >>> Using information criteria *could* be the basis. But I might be
> inclined to
> >>just ignore the issue you describe above though if this only affects one
> study.
> >>>
> >>> Just as a note: It's not necessarily an either-or choice. This model
> is also
> >>possible:
> >>>
> >>> (g3=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | study), data =
> dd))
> >>>
> >>> and profile(g3) suggests that all variance components are identifiable
> -
> >>although of course this is quite overfitted with so little data.
> >>>
> >>> >(g1=rma.mv(yi, vi, random = ~1|lab/study, data = dd))
> >>> >(g2=rma.mv(yi, vi, random = list(~1|lab, ~1|study), data = dd))
> >>> >
> >>> >Thanks,
> >>> >Luke
>

	[[alternative HTML version deleted]]