[R-meta] Multilevel Meta-Analysis Pet-Peese and Power

[James Pustejovsky]

Hi Tori,

Here are two papers on PET-PEESE for multi-level/multi-variate
meta-analysis:
- Fernández-Castilla, B., Declercq, L., Jamshidi, L., Beretvas, S. N.,
Onghena, P., & Van den Noortgate, W. (2021). Detecting selection bias in
meta-analyses with multiple outcomes: a simulation study. The Journal of
Experimental Education, 89(1), 125-144. ttps://
doi.org/10.1080/00220973.2019.1582470
- Rodgers, M. A., & Pustejovsky, J. E. (2021). Evaluating meta-analytic
methods to detect selective reporting in the presence of dependent effect
sizes. Psychological Methods, 26(2), 141. https://doi.org/10.1037/met0000300
pre-print: https://osf.io/preprints/metaarxiv/vqp8u/ supplementary
materials: https://osf.io/5f732/
The supplementary materials for our paper includes example code.

The short answer on PET-PEESE is that you can do this by running a
meta-regression (using rma.mv) that includes a measure of precision as a
predictor. For PET, the predictor would be the standard error (or something
closely related to the SE). For PEESE, the predictor would be the squared
standard error. Depending on what effect size metric you're using, you may
need to make a few tweaks to the predictor, as we explain in our paper.

Regarding power analysis, my colleagues and I have a working paper that
describes power approximations for rma.mv() models, but only for the
overall average effect size:
https://osf.io/preprints/metaarxiv/6tp9y/
As far as I know, there has not yet been work on power analysis for
moderators. One could evaluate power through simulation, though I don't
think there are any readily available tools for doing this.

Kind Regards,
James


On Thu, May 5, 2022 at 9:12 AM Tori Peña <Tori.Pena using stonybrook.edu> wrote:

> I was asked by reviewers to conduct a Pet-Peese analysis for publication
> bias, but I'm not sure how to conduct a Pet-Peese analysis on a rma.mv
> object in metafor. (1) Does anyone know if it is possible to run a
> Pet-Peese analysis of publication bias on a rma.mv object? They also asked
> for power analyses which I could not find out how to do for rma.mv objects
> with moderators. (2) Does anyone know how to conduct a power analysis on a
> rma.mv moderator analysis?
>
> I appreciate any guidance you can provide!
>
> Thanks,
> Tori
>
> --
> *Tori Peña <https://www.linkedin.com/in/toripena/> *(she/her/ella)
> Doctoral Candidate, Cognitive Psychology
> Dept. of Psychology
> Stony Brook University
> Stony Brook, NY 11790-2500
> [image: Stony Brook University logo]
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]