[R-meta] AICc or variance components, which one matters more?

[Luke Martinez]

Hi Wolfgang,

Could you possibly elaborate a bit on "in this case, yes"?

This allows me to better justify g3 or g4 models to my co-authors.

Thanks again,
Luke

On Mon, Nov 15, 2021, 11:19 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> >-----Original Message-----
> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >Sent: Saturday, 13 November, 2021 21:10
> >To: Viechtbauer, Wolfgang (SP)
> >Cc: R meta
> >Subject: Re: [R-meta] AICc or variance components, which one matters more?
> >
> >Interesting! To make sure I'm following you, your suggested g3 model
> >both considers 'study' to be nested in the 'lab', and at the same time
> >it considers 'study' to have its own independent crossed effect. Can
> >we consider the same variable (e.g., study) to be both nested and
> >crossed at the same time?
>
> In this case, yes.
>
> >If so, I can then suggest the following model as well:
> >
> >(g4=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | lab), data = dd))
>
> I suggest you profile the variance components from that model and draw
> appropriate conclusions.
>
> >Doesn't this denote that one is uncertain about whether to take a
> >variable as nested or crossed or there are other justifications?
> >
> >Thank you,
> >Luke
> >
> >On Sat, Nov 13, 2021 at 1:11 PM Viechtbauer, Wolfgang (SP)
> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >>
> >> >-----Original Message-----
> >> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >> >Sent: Saturday, 13 November, 2021 18:44
> >> >To: Viechtbauer, Wolfgang (SP)
> >> >Cc: Philippe Tadger; R meta
> >> >Subject: Re: [R-meta] AICc or variance components, which one matters
> more?
> >> >
> >> >Hi Wolfgang,
> >> >
> >> >I'm fully with you, however, in my data only once 2 labs (labs 1 and
> >> >2) have collaborated on study 2. Specifically, part of study 2 has
> >> >been carried out by lab 1 (one row) and part of it by lab 2 (one row).
> >> >Except in this case, no such between-lab collaborations have ever
> >> >occurred in the data.
> >> >
> >> >If such a between-lab collaboration didn't exist, I could directly go
> >> >for g1 (hierarchical model). But with this collaboration, there is
> >> >just a tiny possibility for g2 (crossed model) as well.
> >> >
> >> >So, do you think AICc should be the basis of the comparison between g1
> >> >vs. g2 or the dominant data structure (ignoring the one exception)?
> >>
> >> Using information criteria *could* be the basis. But I might be
> inclined to
> >just ignore the issue you describe above though if this only affects one
> study.
> >>
> >> Just as a note: It's not necessarily an either-or choice. This model is
> also
> >possible:
> >>
> >> (g3=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | study), data =
> dd))
> >>
> >> and profile(g3) suggests that all variance components are identifiable -
> >although of course this is quite overfitted with so little data.
> >>
> >> >(g1=rma.mv(yi, vi, random = ~1|lab/study, data = dd))
> >> >(g2=rma.mv(yi, vi, random = list(~1|lab, ~1|study), data = dd))
> >> >
> >> >Thanks,
> >> >Luke
>

	[[alternative HTML version deleted]]