[R-meta] Multivariate meta-analysis when "some studies" are multi-outcome

[Viechtbauer, Wolfgang (SP)]

I wouldn't think of dependence in sampling errors arising from 'levels'. It arises whenever multiple estimates are computed with some kind of overlap (or even total overlap) in the subjects being used for the computations.

For example, the exact same n subjects are used to compute estimates 1 and 2 for two different response variables. Or the exact same n subjects are used to compute estimates 1 and 2 for two different follow-up timepoints. Or two estimates are computed, one contrasting treatment group 1 with the control group and the other contrasting treatment group 2 with the control group (then there is overlap due reuse of the control group). One can also have all kinds of combinations of these things.

Of course, the degree of dependence depends on various things: How much overlap is there in the subjects? How strongly are two different response variables correlated? How much does the same response variable correlate over time? Also, the source of the dependence leads to different equations that can be used to estimate the correlation between the sampling errors of non-independent estimates.

But the bottom line is: If at least one subject is involved in the computation of two different estimates, then the sampling errors of the two estimates are (probably) not independent.

Best,
Wolfgang

>-----Original Message-----
>From: Simon Harmel [mailto:sim.harmel using gmail.com]
>Sent: Thursday, 18 March, 2021 13:26
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: [R-meta] Multivariate meta-analysis when "some studies" are multi-
>outcome
>
>Sure, but imagine we have dependence due to the use of multiple treatments from
>the same study (esid), due to the use of multiple outcomes (outcomeid), and
>finally due to the heterogeneity among studies (studyid).
>
>So, here dependence is arising "simultaneously" due to all three levels.  So how
>should one define cluster id in 'impute_covariance_matrix()'?
>
>Best,
>Simon
>
>On Thu, Mar 18, 2021, 7:12 AM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>Whether this makes sense or not depends on how we believe covariances among the
>sampling errors are arising. Two estimates from the same study based on the same
>sample of subjects (e.g., based on two different response variables) probably have
>correlated sampling errors. Two estimates from the same study, one for female, the
>other for male participants, not (the underlying true effects may still be
>correlated). So, the 'cluster' variable should be specified accordingly (i.e.,
>same levels for the two estimates in the first case, different levels for the two
>estimates in the second case; i.e., 1, 1, 2, 3).
>
>>-----Original Message-----
>>From: Simon Harmel [mailto:sim.harmel using gmail.com]
>>Sent: Thursday, 18 March, 2021 12:53
>>To: Viechtbauer, Wolfgang (SP)
>>Cc: R meta
>>Subject: Re: [R-meta] Multivariate meta-analysis when "some studies" are multi-
>>outcome
>>
>>Dear Wolfgang,
>>
>>Many thanks for your response. The reason I asked which level of dependence does
>V
>>matrix account for was that I realized (at least when using
>>'impute_covariance_matrix()' function) that always the highest cluster level
>>(e.g., study_id rather than outcome_id or es_id) is used to construct the V
>>matrix.
>>
>>So, is there a reason for that?
>>
>>Many thanks
>>
>>On Thu, Mar 18, 2021, 6:38 AM Viechtbauer, Wolfgang (SP)
>><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>Dear Simon,
>>
>>Roughly, whatever you put into 'random' accounts for heterogeneity in the true
>>effects (at possibly multiple levels) and can account for possible dependencies
>in
>>these true effects. Whatever you put into V accounts for the sampling variances
>in
>>the estimates or more precisely, their sampling errors, and can account for
>>possible dependencies in these sampling errors.
>>
>>I use the term 'dependencies' in a very vague/broad sense here, since such
>>dependencies (in the true effects and/or the sampling errors) can arise for all
>>kinds of different reasons.
>>
>>Best,
>>Wolfgang
>>
>>>-----Original Message-----
>>>From: Simon Harmel [mailto:sim.harmel using gmail.com]
>>>Sent: Wednesday, 17 March, 2021 18:01
>>>To: Viechtbauer, Wolfgang (SP)
>>>Cc: Gladys Barragan-Jason; R meta
>>>Subject: Re: [R-meta] Multivariate meta-analysis when "some studies" are multi-
>>>outcome
>>>
>>>Dear Wolfgang,
>>>
>>>I do want to quickly follow-up on the answer you linked
>>>(https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-July/000896.html).
>>>
>>>In `rma.mv(y ~ x1 + x2, V, random = ~ 1 | study/outcome/id, data=data)`, we
>>>apparently take into account dependence among effect sizes due to multiple
>>>treatments (`id`), and multiple outcomes (`outcome`) by means of using a level
>>for
>>>each.
>>>
>>>If so, what is the role of `V` when it comes to accounting for effect
>>>size dependency? Does `V` simply determine the pair-wise structure of effect
>size
>>>dependency? If yes, at what level?
>>>
>>>Simon