[R-meta] overall efect size from subgroup analysis

[James Pustejovsky]

Following up on Wolfgang's comments, I think there is one caveat to the
method he's suggested. Doing a fixed effect meta-analysis of the sub-group
average effects will work if the subgroup effects are based on
non-overlapping sets of studies. For example, the following:

Subgroup A Study 1
           Study 3
           Study 4
Subgroup B Study 5
           Study 6
           Study 7
Subgroup C Study 8
           Study 9
           Study 10
           Study 11
           Study 12

However, if the subgroups are comprised of different outcomes from
overlapping sets of studies, then the subgroup average effects are not
independent. Here's an illustration of a data structure with overlapping
subgroups. X indicates that a given study includes an effect size from that
subgroup.

          Subgroup A   Subgroup B   Subgroup C
Study 1        X            X            X
Study 2                     X            X
Study 3        X
Study 4        X                         X
Study 5        X            X            X
Study 6        X            X
Study 7        X                         X
Study 8                                  X

If this is the case, then pooling across subgroups using fixed effect
meta-analysis will give inaccurate standard errors (that will likely be
smaller than the true degree of uncertainty). I don't think there would be
any way to calculate accurate standard errors unless you have access to the
raw effect size data.

James


On Wed, Apr 28, 2021 at 4:48 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Diego,
>
> If these are 95% CIs computed based on a normal distribution, then we can
> back-calculate the standard errors with:
>
> yi <- c(-48, -30, -15)
> ci.lb <- c(-73, -38, -24)
> ci.ub <- c(-12, -21, -6)
> sei <- (ci.ub - ci.lb) / (2*1.96)
>
> We can check if this yields the same bounds with:
>
> round(yi - 1.96 * sei)
> round(yi + 1.96 * sei)
>
> This doesn't seem to work out for the first group, but for the second and
> third (the upper bound of -22 for the second group instead of -21 could
> just be a rounding error). Either there is a typo for the first group or
> the CI was obtained in a different way (as you mentioned bootstrapping in
> your first post). Are you sure it says -12 for the upper bound of the first
> group and not -22? Because then it would be fairly close again.
>
> Ignoring this isse, you now have the estimates and corresponding SEs, so
> you can now pool them with:
>
> library(metafor)
> rma(yi, sei=sei, method="FE")
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >Behalf Of Diego Grados Bedoya
> >Sent: Wednesday, 28 April, 2021 9:10
> >To: Michael Dewey
> >Cc: R meta
> >Subject: Re: [R-meta] overall efect size from subgroup analysis
> >
> >Dear Michael,
> >
> >Sorry, I did not notice that the email was mangled.
> >
> >Thank for the link but I do not have access to the original data of the
> >subgroups. I just have the effect sizes, CIs and number of comparisons
> >(observations).
> >
> >subgroup CI_lower_limit effect_size CI_upper_limit number_comparisons
> >group_1 -73 -48 -12 5
> >group_2 -38 -30 -21 43
> >group_3 -24 -15 -6 73
> >
> >Greetings,
> >
> >Diego
> >
> >On Tue, 27 Apr 2021 at 16:08, Michael Dewey <lists using dewey.myzen.co.uk>
> wrote:
> >
> >> Dear Diego
> >>
> >> I think you may find
> >>
> https://metafor-project.org/doku.php/tips:comp_two_independent_estimates
> >> helpful.
> >>
> >> Your data is close to unreadable as you posted in HTML and that becomes
> >> mangled. If you need to re-post you have to find a way to send plain
> >> text. I do not use Gmail but I am sure it is possible.
> >>
> >> Michael
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]