[R-meta] (Too) Many effect sizes for one single group

[Lukasz Stasielowicz]

Dear Catia,

aggregating (computing average effect for a particular study) could 
conceal heterogeneity between effect sizes (e.g. r1 = -.20, r2= +.20 --> 
r_mean = 0). It could also impact moderator analyses (less effect sizes 
to test moderators that explain within-study variability).

Of course one could also directly test the impact of aggregating on 
heterogeneity estimates (e.g. tau), but in general I would suggest 
keeping as many effect sizes as possible by using other sensitivity 
analyses (see previous message).


Best wishes
Lukasz
-- 
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstraße 20
49074 Osnabrück (Germany)

Am 08.01.2022 um 12:00 schrieb r-sig-meta-analysis-request using r-project.org:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis using r-project.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request using r-project.org
> 
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner using r-project.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
> 
> 
> Today's Topics:
> 
>     1. Re: (Too) Many effect sizes for one single group
>        (Lukasz Stasielowicz)
>     2. Re: (Too) Many effect sizes for one single group
>        (=?UTF-8?Q?C=C3=A1tia_Ferreira_De_Oliveira?=)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Fri, 7 Jan 2022 16:38:58 +0100
> From: Lukasz Stasielowicz <lukasz.stasielowicz using uni-osnabrueck.de>
> To: r-sig-meta-analysis using r-project.org
> Cc: cmfo500 using york.ac.uk
> Subject: Re: [R-meta] (Too) Many effect sizes for one single group
> Message-ID: <61206fc1-c2d1-2f92-82f7-863b4d8a3576 using uni-osnabrueck.de>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> Dear Catia,
> 
> under certain circumstances it could be a valid concern.
> Fortunately, one can test it directly. One could conduct a sensitivity
> analysis to examine the impact in your specific case: Do the results
> (mean effect, standard error etc.) change much if you exclude certain
> effect sizes?
> 
> Example:
> Scenario 1: All effects are considered
> Scenario 2: The study with "too many" effect sizes is excluded
> Scenario 3: Only one or several effect sizes from the problematic study
> are considered, e.g. by using the sample() function and choosing a
> certain number of effects randomly. One could also repeat this procedure
> to check the influence of the selection procedure.
> 
> If the estimates differ only slightly across the analyses then you could
> proceed with the original idea (including all effects). You could
> mention in the report that this decision is based on some sensitivity
> analyses that you've conducted.
> 
> 
> Best wishes
> Lukasz