[R-meta] fixed-effect multivariate model interpretation

[Viechtbauer, Wolfgang (SP)]

See for example the Gleser & Olkin chapter:

Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 357–376). New York: Russell Sage Foundation.

https://www.metafor-project.org/doku.php/analyses:gleser2009

Best,
Wolfgang

>-----Original Message-----
>From: Filippo Gambarota [mailto:filippo.gambarota using gmail.com]
>Sent: Monday, 03 January, 2022 18:02
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: [R-meta] fixed-effect multivariate model interpretation
>
>Ah great! do you have any references for this? because I would like to clearly
>understand what is going on. Cochrane's Q-statistic is the deviation of each
>study from the estimated average effect weighted by the precision. In this case,
>the "average" effect is the average between outcomes? Or the formula is
>different?
>Thank you!
>
>On Mon, 3 Jan 2022 at 17:12, Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>It's just the multivariate version of Cochrane's Q-test. It does not estimate a
>random-effects model. It simply tests whether the observed amount of variability
>is larger than expected based on the sampling variances (and their covariances
>when V includes those) and any moderators specified.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Filippo Gambarota [mailto:filippo.gambarota using gmail.com]
>>Sent: Monday, 03 January, 2022 17:11
>>To: Viechtbauer, Wolfgang (SP)
>>Cc: R meta
>>Subject: Re: [R-meta] fixed-effect multivariate model interpretation
>>
>>Thank you Wolfgang!
>>So my related question is how this residual heterogeneity is estimated in order
>>to compute the Q statistic? Because if the model is still estimating and testing
>>the presence of heterogeneity, from a multivariate model I would have expected
>>one residual heterogeneity term for each outcome (the same as I have one tau per
>>outcome if I fit the random-effect version).
>>
>>On Mon, 3 Jan 2022 at 16:50, Viechtbauer, Wolfgang (SP)
>><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>Hi Filippo,
>>
>>You can *assume* that there is no residual heterogeneity, but there may be. That
>>is what the test of residual heterogeneity is testing here (whether your
>>assumption is correct or not).
>>
>>Best,
>>Wolfgang
>>
>>>-----Original Message-----
>>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>>Behalf Of Filippo Gambarota
>>>Sent: Monday, 03 January, 2022 16:42
>>>To: R meta
>>>Subject: [R-meta] fixed-effect multivariate model interpretation
>>>
>>>Hello!
>>>I'm fitting for the first time a multivariate fixed-effect model using
>>>metafor. The code is:
>>>
>>>```
>>>rma.mv(yi, V, mods = ~ 0 + outcome, data = data, test = "t")
>>>```
>>>Where V is the block variance-covariance matrix created with vcalc()
>>>that represents the covariance between different outcome levels within
>>>each study. The outcome is a factor that represents different effect
>>>sizes measured on the same participants within a study.
>>>The model as expected did not estimate tau for each outcome and test
>>>all coefficients (each outcome mean with this parametrization) against
>>>0 (both the omnibus test and each beta). My question is about the
>>>*residual heterogeneity* parameter and the associated Q test. Under
>>>this model, I should have assumed that there is no heterogeneity
>>>within each outcome level so I'm not sure how to interpret the
>>>residual heterogeneity in this case.
>>>Thank you!
>>>Filippo
>>>
>>>--
>>>Filippo Gambarota
>>>PhD Student - University of Padova
>>>Department of Developmental and Social Psychology
>>>Website: filippogambarota.netlify.app
>>>Research Group: Colab   Psicostat