[R-meta] AICc or variance components, which one matters more?

[Luke Martinez]

Ok, I see what you mean regarding g4. "lab" is just repetitive
(profile curves for two repetitive lab var. components are mainly
flat). Ok g4 is out.

 But I still wonder about the thinking behind g3 where 'study' can be
both nested and at the same time crossed?

(g3=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | study), data = dd))

On Mon, Nov 15, 2021 at 12:07 PM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>
> Please post the profile likelihood plots for the three variance components of model g4.
>
> >-----Original Message-----
> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >Sent: Monday, 15 November, 2021 18:37
> >To: Viechtbauer, Wolfgang (SP)
> >Cc: R meta
> >Subject: Re: [R-meta] AICc or variance components, which one matters more?
> >
> >In g4, also, all variance components are identifiable (and larger in magnitude).
> >
> >But what's the thinking behind either g3 or g4 where the same grouping variable
> >is both nested and crossed?
> >
> >On Mon, Nov 15, 2021, 11:33 AM Viechtbauer, Wolfgang (SP)
> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >In g3, all variance components are identifiable.
> >
> >What have you found out about g4?
> >
> >>-----Original Message-----
> >>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >>Sent: Monday, 15 November, 2021 18:28
> >>To: Viechtbauer, Wolfgang (SP)
> >>Cc: R meta
> >>Subject: Re: [R-meta] AICc or variance components, which one matters more?
> >>
> >>Hi Wolfgang,
> >>
> >>Could you possibly elaborate a bit on "in this case, yes"?
> >>
> >>This allows me to better justify g3 or g4 models to my co-authors.
> >>
> >>Thanks again,
> >>Luke
> >>
> >>On Mon, Nov 15, 2021, 11:19 AM Viechtbauer, Wolfgang (SP)
> >><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >>>-----Original Message-----
> >>>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >>>Sent: Saturday, 13 November, 2021 21:10
> >>>To: Viechtbauer, Wolfgang (SP)
> >>>Cc: R meta
> >>>Subject: Re: [R-meta] AICc or variance components, which one matters more?
> >>>
> >>>Interesting! To make sure I'm following you, your suggested g3 model
> >>>both considers 'study' to be nested in the 'lab', and at the same time
> >>>it considers 'study' to have its own independent crossed effect. Can
> >>>we consider the same variable (e.g., study) to be both nested and
> >>>crossed at the same time?
> >>
> >>In this case, yes.
> >>
> >>>If so, I can then suggest the following model as well:
> >>>
> >>>(g4=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | lab), data = dd))
> >>
> >>I suggest you profile the variance components from that model and draw
> >>appropriate conclusions.
> >>
> >>>Doesn't this denote that one is uncertain about whether to take a
> >>>variable as nested or crossed or there are other justifications?
> >>>
> >>>Thank you,
> >>>Luke
> >>>
> >>>On Sat, Nov 13, 2021 at 1:11 PM Viechtbauer, Wolfgang (SP)
> >>><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >>>>
> >>>> >-----Original Message-----
> >>>> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
> >>>> >Sent: Saturday, 13 November, 2021 18:44
> >>>> >To: Viechtbauer, Wolfgang (SP)
> >>>> >Cc: Philippe Tadger; R meta
> >>>> >Subject: Re: [R-meta] AICc or variance components, which one matters more?
> >>>> >
> >>>> >Hi Wolfgang,
> >>>> >
> >>>> >I'm fully with you, however, in my data only once 2 labs (labs 1 and
> >>>> >2) have collaborated on study 2. Specifically, part of study 2 has
> >>>> >been carried out by lab 1 (one row) and part of it by lab 2 (one row).
> >>>> >Except in this case, no such between-lab collaborations have ever
> >>>> >occurred in the data.
> >>>> >
> >>>> >If such a between-lab collaboration didn't exist, I could directly go
> >>>> >for g1 (hierarchical model). But with this collaboration, there is
> >>>> >just a tiny possibility for g2 (crossed model) as well.
> >>>> >
> >>>> >So, do you think AICc should be the basis of the comparison between g1
> >>>> >vs. g2 or the dominant data structure (ignoring the one exception)?
> >>>>
> >>>> Using information criteria *could* be the basis. But I might be inclined to
> >>>just ignore the issue you describe above though if this only affects one study.
> >>>>
> >>>> Just as a note: It's not necessarily an either-or choice. This model is also
> >>>possible:
> >>>>
> >>>> (g3=rma.mv(yi, vi, random = list(~1|lab/study, ~ 1 | study), data = dd))
> >>>>
> >>>> and profile(g3) suggests that all variance components are identifiable -
> >>>although of course this is quite overfitted with so little data.
> >>>>
> >>>> >(g1=rma.mv(yi, vi, random = ~1|lab/study, data = dd))
> >>>> >(g2=rma.mv(yi, vi, random = list(~1|lab, ~1|study), data = dd))
> >>>> >
> >>>> >Thanks,
> >>>> >Luke

-------------- next part --------------
A non-text attachment was scrubbed...
Name: g4.PNG
Type: image/png
Size: 79918 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20211115/d838a379/attachment-0001.png>