[R-meta] Advice on running a multilevel meta-analysis

[Viechtbauer, Wolfgang (NP)]

Dear Matheus,

With respect to shared control groups:

You did not mention what kind of effect size measure you are using, but the chapter on dependent effect sizes by Gleser and Olkin (2009) in 'The handbook of research synthesis and meta-analysis (2nd ed.)' is a good start and covers various measures (risk differences, risk/odds ratios, the difference of arcsine square-root transformed proportions, and standardized mean differences). See also:

https://www.metafor-project.org/doku.php/analyses:gleser2009

In essence, you need to compute the covariance between estimates that share a control group. The vcalc() function from metafor can also help with this:

https://wviechtb.github.io/metafor/reference/vcalc.html

With respect to multiple correlated outcomes per experiment:

Again, one should in principle calculate the covariance between such estimates. This is often difficult in practice due to lack of the information needed to compute to covariance. In any case, a multilevel/multivariate model is then typically used to account for dependencies in addition to using cluster-robust standard errors. See:

https://wviechtb.github.io/metafor/reference/misc-recs.html#general-workflow-for-meta-analyses-involving-complex-dependency-structures

A good reference here is:

Pustejovsky, J. E., & Tipton, E. (2022). Meta-analysis with robust variance estimation: Expanding the range of working models. Prevention Science, 23, 425-438. https://doi.org/10.1007/s11121-021-01246-3

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Matheus Gallas Lopes via R-sig-meta-analysis
> Sent: Monday, March 10, 2025 14:50
> To: R sig meta analysis <r-sig-meta-analysis using r-project.org>
> Cc: Matheus Gallas Lopes <matheus.lopes using ufrgs.br>
> Subject: [R-meta] Advice on running a multilevel meta-analysis
>
> Hi everyone,
>
> We're conducting a systematic review and meta-analysis on the effects of
> NMDA receptor antagonists on social behavior in animals (PROSPERO:
> CRD42023402129). Our goal is to assess how these antagonists affect
> social behavior and whether antipsychotics can counteract these effects.
>
> Since this is our first time running a multilevel meta-analysis, we want
> to ensure we properly account for dependencies within experiments and
> within studies. We have a few methodological questions:
>
> Shared control groups within experiments: Some studies report multiple
> treatment groups (e.g., different doses of an NMDA antagonist) that
> share the same control group. In past reviews, we have split the control
> sample size across comparisons, but we know this approach has
> limitations. What is the best way to properly account for shared
> controls in a multilevel model?
>
> Multiple correlated outcomes per experiment: As per our protocol, our
> goal was to extract all relevant outcomes measured in the same animals
> related to social behavior and combine them, rather than selecting just
> one. Many studies report multiple measures (e.g., time spent interacting
> and number of interactions), and we want to properly aggregate them in
> our multilevel meta-analysis to avoid redundancy while preserving
> information. What would be the best approach for this?
>
> If anyone has experience handling these dependencies or can suggest
> useful references, we'd really appreciate your insights!
>
> Thanks in advance!
>
> Best,
> Matheus
>
> --
> Matheus Gallas-Lopes
> PhD student in Pharmacology and Therapeutics
> Laboratório de Neurobiologia e Psicofarmacologia Experimental
> (PsychoLab)
> Universidade Federal do Rio Grande do Sul (UFRGS)
> Currículo Lattes: https://goo.gl/iwHPAM
> ORCID: https://goo.gl/Tp2S6w
> ResearchGate: https://goo.gl/lja63C
>
> matheus.lopes using ufrgs.br