[R-meta] (Too) Many effect sizes for one single group

Lukasz Stasielowicz |uk@@z@@t@@|e|ow|cz @end|ng |rom un|-o@n@brueck@de
Mon Jan 10 12:13:55 CET 2022

Dear Catia,

aggregating (computing average effect for a particular study) could 
conceal heterogeneity between effect sizes (e.g. r1 = -.20, r2= +.20 --> 
r_mean = 0). It could also impact moderator analyses (less effect sizes 
to test moderators that explain within-study variability).

Of course one could also directly test the impact of aggregating on 
heterogeneity estimates (e.g. tau), but in general I would suggest 
keeping as many effect sizes as possible by using other sensitivity 
analyses (see previous message).

Best wishes
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstraße 20
49074 Osnabrück (Germany)

Am 08.01.2022 um 12:00 schrieb r-sig-meta-analysis-request using r-project.org:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis using r-project.org
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request using r-project.org
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner using r-project.org
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
> Today's Topics:
>     1. Re: (Too) Many effect sizes for one single group
>        (Lukasz Stasielowicz)
>     2. Re: (Too) Many effect sizes for one single group
>        (=?UTF-8?Q?C=C3=A1tia_Ferreira_De_Oliveira?=)
> ----------------------------------------------------------------------
> Message: 1
> Date: Fri, 7 Jan 2022 16:38:58 +0100
> From: Lukasz Stasielowicz <lukasz.stasielowicz using uni-osnabrueck.de>
> To: r-sig-meta-analysis using r-project.org
> Cc: cmfo500 using york.ac.uk
> Subject: Re: [R-meta] (Too) Many effect sizes for one single group
> Message-ID: <61206fc1-c2d1-2f92-82f7-863b4d8a3576 using uni-osnabrueck.de>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> Dear Catia,
> under certain circumstances it could be a valid concern.
> Fortunately, one can test it directly. One could conduct a sensitivity
> analysis to examine the impact in your specific case: Do the results
> (mean effect, standard error etc.) change much if you exclude certain
> effect sizes?
> Example:
> Scenario 1: All effects are considered
> Scenario 2: The study with "too many" effect sizes is excluded
> Scenario 3: Only one or several effect sizes from the problematic study
> are considered, e.g. by using the sample() function and choosing a
> certain number of effects randomly. One could also repeat this procedure
> to check the influence of the selection procedure.
> If the estimates differ only slightly across the analyses then you could
> proceed with the original idea (including all effects). You could
> mention in the report that this decision is based on some sensitivity
> analyses that you've conducted.
> Best wishes
> Lukasz

More information about the R-sig-meta-analysis mailing list