[R-meta] adjusting individual studies for publication bias

[Filippo Gambarota]

Thank you Wolfgang for the correction about the p value. And yes, about the
idea of correcting the individual studies you are right. I was thinking
something related to the type-m error
https://library.virginia.edu/data/articles/assessing-type-s-and-type-m-errors
or something where a study contribute with a certain weight to an inflated
estimated effect size thus we could remove proportionally the inflated part
to obtain a set of studies where the weighted average is the pb corrected
parameter.
Thank you again!
Filippo

On Thu, May 8, 2025 at 3:06 PM Viechtbauer, Wolfgang (NP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Hi Filippo,
>
> First, a general note about your code: The p-values you get from
> summary(sim) are two-sided, but in selmodel(), you are using the (default)
> alternative="greater". If you use selmodel(res, type = "negexp",
> alternative = "two.sided"), then you will find that the delta=5 value can
> be recovered quite accurately.
>
> As for your actual question: Your idea of using
>
> sel$b[[1]] + rnorm(sel$k, 0, sqrt(sel$se^2 + sel$vi + sel$tau2))
>
> is sensible, but this is more like a new sample from the same population
> (with identical sampling variances) than adjusting the actually observed
> estimates. But publication bias does not change the effects themselves - it
> changes *which* estimates we end up seeing in our sample. So I am not sure
> to what extent the idea of 'adjusting the observed effects for publication
> bias' makes sense in the first place.
>
> Best,
> Wolfgang
>
> > -----Original Message-----
> > From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org>
> On Behalf
> > Of Filippo Gambarota via R-sig-meta-analysis
> > Sent: Monday, May 5, 2025 11:53
> > To: r-sig-meta-analysis using r-project.org
> > Cc: Filippo Gambarota <filippo.gambarota using unipd.it>
> > Subject: [R-meta] adjusting individual studies for publication bias
> >
> > Hi!
> > I was wondering if there is a method, maybe using selection models, to
> > adjust the individual studies taking into account the publication bias. I
> > was thinking something like:
> >
> > ```
> > library(metafor)
> >
> > k <- 500
> > tau2 <- 0.1
> > d <- 0.2
> > n1 <- n2 <- round(runif(k, 10, 300))
> > deltai <- rnorm(k, 0, sqrt(tau2))
> > yi <- rnorm(k, d, sqrt(1/n1 + 1/n2 + tau2))
> > vi <- (rchisq(k, n1 + n2 - 2) / (n1 + n2 - 2)) * (1/n1 + 1/n2)
> >
> > sim <- data.frame(yi, vi)
> > sim <- escalc(yi = yi, vi = vi, data = sim)
> > sim <- summary(sim)
> > wi <- exp(-5 * sim$pval) # simulating selecting with negexp model
> > keep <- rbinom(nrow(sim), 1, wi) == 1 # keep if coin flip is 1
> > sims <- sim[keep, ]
> >
> > res <- rma(yi, vi, data = sims)
> > sel <- selmodel(res, type = "negexp")
> >
> > # linear combination of b + ranef
> >
> > yi_adj <- sel$b[[1]] + rnorm(sel$k, 0, sqrt(sel$se^2 + sel$vi +
> sel$tau2))
> >
> > plot(sims$yi, yi_adj)
> > ```
> > But of course i'm not sure. Something like using b, tau2, vi and seb to
> > estimate the individual effect correcting for publication bias.
> > Thank you!
> > --
> > *Filippo Gambarota, PhD*
> > Postdoctoral Researcher - University of Padova
> > Department of Developmental and Social Psychology
> > Website: filippogambarota.github.io
> > Research Groups: Psicostat <https://psicostat.dpss.psy.unipd.it/>
>


-- 
*Filippo Gambarota, PhD*
Postdoctoral Researcher - University of Padova
Department of Developmental and Social Psychology
filippogambarota.github.io | Psicostat
<https://psicostat.dpss.psy.unipd.it/>

	[[alternative HTML version deleted]]