[R-meta] (Too) Many effect sizes for one single group

[Lukasz Stasielowicz]

Dear Catia,

under certain circumstances it could be a valid concern.
Fortunately, one can test it directly. One could conduct a sensitivity 
analysis to examine the impact in your specific case: Do the results 
(mean effect, standard error etc.) change much if you exclude certain 
effect sizes?

Example:
Scenario 1: All effects are considered
Scenario 2: The study with "too many" effect sizes is excluded
Scenario 3: Only one or several effect sizes from the problematic study 
are considered, e.g. by using the sample() function and choosing a 
certain number of effects randomly. One could also repeat this procedure 
to check the influence of the selection procedure.

If the estimates differ only slightly across the analyses then you could 
proceed with the original idea (including all effects). You could 
mention in the report that this decision is based on some sensitivity 
analyses that you've conducted.


Best wishes
Lukasz
-- 
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstraße 20
49074 Osnabrück (Germany)

Am 06.01.2022 um 12:00 schrieb r-sig-meta-analysis-request using r-project.org:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis using r-project.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request using r-project.org
> 
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner using r-project.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
> 
> 
> Today's Topics:
> 
>     1. (Too) Many effect sizes for one single group
>        (=?UTF-8?Q?C=C3=A1tia_Ferreira_De_Oliveira?=)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 5 Jan 2022 19:32:29 +0000
> From: =?UTF-8?Q?C=C3=A1tia_Ferreira_De_Oliveira?= <cmfo500 using york.ac.uk>
> To: R meta <r-sig-meta-analysis using r-project.org>,  "Viechtbauer,
> 	Wolfgang (SP)" <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Subject: [R-meta] (Too) Many effect sizes for one single group
> Message-ID:
> 	<CACw+TffviaNdFERh3Pp361ZJVO2F11WG7S2G8WdWcONiHcusFg using mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Dear Wolfgang,
> 
> I hope you had a lovely start to the year.
> I am sorry for starting the year with questions, but I just wanted to check
> whether there is any drawback from including a lot of effect sizes from a
> single paper when most labs contributed to the meta-analysis with just one
> or two effect sizes? This resulted in a dataset where half of the effect
> sizes come from multiple experiments run by the same group. The nested
> nature of the data and dependency of some effect sizes coming from the same
> participants is acknowledged in the model.
> 
> Thank you!
> 
> Catia
>