[R-meta] Multivariate meta-analysis when "some studies" are multi-outcome

[Viechtbauer, Wolfgang (SP)]

Dear Gladys,

random=list( ~1|study,~1|lab) makes little sense, since study = 1 from lab X would receive the same random effect as study = 1 from lab Y, but I assume that the 1 is just a number that distinguishes multiple studies from the same lab and has no inherent meaning that somehow links the 1 from lab X to the 1 from lab Y.

Instead, random=~1|lab/study (you don't need the list()) seems more applicable. This adds random effects for each level of lab and for each study within each lab. Hence, the combination X-1 will be a different random effect than the combiantion Y-1.

However, this assumes no heterogeneity of the true effects within a study (within a lab). For example, this would assume that the 0.3 and 0.6 (for X-1) are both estimates of the same underlying true effect, which is assumed to be identical. That may or may not be true. Hence, as I noted in my response to Simon (see also https://www.metafor-project.org/doku.php/analyses:konstantopoulos2011 and search the archives for similar discussions), one should add higher level random effects (such as those for study and lab) to the random effects at the estimate level. So, if you dataset is called 'dat', then do:

dat$estid <- 1:nrow(dat)

in combination with random=~1|lab/study/estid.

Finally, random=list(~1|lab, ~1|lab/study) is superfluous. ~ 1 | lab adds random effects for each level of lab and ~1|lab/study also does this (plus it adds random effects for each level of study within lab). So this would add random effects for lab twice.

Best,
Wolfgang

>-----Original Message-----
>From: Gladys Barragan-Jason [mailto:gladou86 using gmail.com]
>Sent: Thursday, 18 March, 2021 11:17
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: [R-meta] Multivariate meta-analysis when "some studies" are multi-
>outcome
>
>Dear Wolfgang,
>
>After reading your post carefully, I am still a bit confused about how to
>implement the random effects.
>My data are coded as follows (similar number or text in one lab means same lab and
>similar number or text in one study means same participants). As you can see you
>can have several studies from one lab and several effect sizes for one study.
>Should I then code the random effects like this:
>random=list( ~1|study,~1|lab)
>or like this:
>random=list( ~1|lab/study)
>or like that;
>random=list( ~1|lab, ~1|lab/study)
>
>lab       study   effect size
>X              1           0.3
>X              1           0.6
>X              2          0.2
>Y              1           0.5
>Y              2           0.1
>Z              1           0.1
>
>Thanks a lot for your help.
>Best wishes,
>Gladys
>
>Le mer. 17 mars 2021 à 13:27, Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> a écrit :
>Dear Gladys,
>
>Whether this makes sense depends on how thse variables are coded. There have been
>several posts in the past on this mailing list where this was discussed. One that
>I quickly found is:
>
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-July/000896.html
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Gladys Barragan-Jason [mailto:gladou86 using gmail.com]
>>Sent: Tuesday, 16 March, 2021 11:39
>>To: Viechtbauer, Wolfgang (SP)
>>Cc: Simon Harmel; R meta
>>Subject: Re: [R-meta] Multivariate meta-analysis when "some studies" are multi-
>>outcome
>>
>>Dear Wolfgang,
>>
>>Following Simon's question, I am also comparing the efficiency of programs (pre-
>>post comparisons).
>>For some of them, I do have several effect sizes for one study and one lab. So I
>>was using the following code to account for it.
>>
>>res.ExpNC<-rma.mv(yi, vi, mods= ~ categ , random=list(
>~1|study,~1|lab),data=dat2)
>>
>>But I am now wondering whether I should do this instead:
>>
>>dat2$estid <- 1:nrow(dat2)
>>res.ExpNC<-rma.mv(yi, vi, mods= ~ categ , random=list(
>>~1|study/estid,~1|lab/estid),data=dat2)
>>
>>What do you think?
>>
>>Thanks a lot for your response,
>>
>>Gladys
>>
>>Le mar. 16 mars 2021 à 11:28, Viechtbauer, Wolfgang (SP)
>><wolfgang.viechtbauer using maastrichtuniversity.nl> a écrit :
>>Dear Simon,
>>
>>At the very least, you should add random effects at the level of the studies and
>>at the level of the estimates, so:
>>
>>dat$estid <- 1:nrow(dat)
>>
>>and then
>>
>>random = ~ 1 | id / estid
>>
>>For longitudinal data, one could also consider using some kind of autocorrelation
>>structure for the estimates within studies. There are some examples here:
>>
>>https://wviechtb.github.io/metafor/reference/dat.ishak2007.html
>>https://wviechtb.github.io/metafor/reference/dat.fine1993.html
>>
>>clubSandwich::impute_covariance_matrix() also allows for the construction of a V
>>matrix with an autocorrelation structure.
>>
>>If the different outcomes are meaningfully related across studies (i.e., outcome
>>'1' stands for the same thing across all studies), then one could also consider
>>using an unstructured var-cov matrix with correlated random effects for outcomes
>>within studies. This would be akin to:
>>
>>https://www.metafor-project.org/doku.php/analyses:berkey1998
>>
>>Best,
>>Wolfgang