[R-meta] Confidence Intervals after aggregating estimates

[Maximilian Steininger]

Dear all,

I want to create a forest plot for a multilevel meta-analysis. I ran a three-level metaanalysis that accounts for different dependencies in the data (as suggested here https://wviechtb.github.io/metafor/reference/misc-recs.html#general-workflow-for-meta-analyses-involving-complex-dependency-structures). I am under the impression that it „is more informative“ to create a forest plot for this analysis on the level of studies (using aggregated effect sizes if a study has more than one) rather than on the level of individual effects. I followed the example code from the metafor-project, which can be found here:

https://www.metafor-project.org/doku.php/tips:forest_plot_with_aggregated_values

However, I noticed some peculiarities regarding the confidence intervals of the effect sizes in the resulting forest plot, which I can’t make sense of.

Referring to the dataset and the plots presented in the link above, I noticed that after aggregating the data, the confidence intervals in the plot get very large. Even so, for studies that only have one effect size estimate. E.g. before aggregating the dataset studies 4, 5, and 8 in the example have an effect size and CIs of:

Study 4: -0.04 [-0.40, 0.31]
Study 5: 1.55 [0.82, 2.28]
Study 8: 0.37 [0.09, 0.65] 

Applying the aggregate() function to the dataset, recalculating the model - which indeed results in the same pooled estimate - and creating the forest plot with the aggregated data works, but the studies have rather large CIs. Studies 4, 5, and 8 - which only have one effect estimate - now have a considerably larger CI e.g.:

Study 4: -0.04 [-1.06, 0.97]
Study 5: 1.55 [0.35, 2.75]
Study 8: 0.37 [-0.62, 1.36]

I understand that aggregating the data of studies with several effect sizes can lead to larger CIs, but why is this also happening for studies with single effect estimates?

Many thanks!
Max
 
——

Mag. Maximilian Steininger
  PhD candidate

  Social, Cognitive and Affective Neuroscience Unit
  Faculty of Psychology
  University of Vienna

  Liebiggasse 5
  1010 Vienna, Austria

  e: maximilian.steininger using univie.ac.at
  w: http://scan.psy.univie.ac.at