[R-meta] adjusting individual studies for publication bias

[Viechtbauer, Wolfgang (NP)]

Hi Filippo,

First, a general note about your code: The p-values you get from summary(sim) are two-sided, but in selmodel(), you are using the (default) alternative="greater". If you use selmodel(res, type = "negexp", alternative = "two.sided"), then you will find that the delta=5 value can be recovered quite accurately.

As for your actual question: Your idea of using

sel$b[[1]] + rnorm(sel$k, 0, sqrt(sel$se^2 + sel$vi + sel$tau2))

is sensible, but this is more like a new sample from the same population (with identical sampling variances) than adjusting the actually observed estimates. But publication bias does not change the effects themselves - it changes *which* estimates we end up seeing in our sample. So I am not sure to what extent the idea of 'adjusting the observed effects for publication bias' makes sense in the first place.

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Filippo Gambarota via R-sig-meta-analysis
> Sent: Monday, May 5, 2025 11:53
> To: r-sig-meta-analysis using r-project.org
> Cc: Filippo Gambarota <filippo.gambarota using unipd.it>
> Subject: [R-meta] adjusting individual studies for publication bias
>
> Hi!
> I was wondering if there is a method, maybe using selection models, to
> adjust the individual studies taking into account the publication bias. I
> was thinking something like:
>
> ```
> library(metafor)
>
> k <- 500
> tau2 <- 0.1
> d <- 0.2
> n1 <- n2 <- round(runif(k, 10, 300))
> deltai <- rnorm(k, 0, sqrt(tau2))
> yi <- rnorm(k, d, sqrt(1/n1 + 1/n2 + tau2))
> vi <- (rchisq(k, n1 + n2 - 2) / (n1 + n2 - 2)) * (1/n1 + 1/n2)
>
> sim <- data.frame(yi, vi)
> sim <- escalc(yi = yi, vi = vi, data = sim)
> sim <- summary(sim)
> wi <- exp(-5 * sim$pval) # simulating selecting with negexp model
> keep <- rbinom(nrow(sim), 1, wi) == 1 # keep if coin flip is 1
> sims <- sim[keep, ]
>
> res <- rma(yi, vi, data = sims)
> sel <- selmodel(res, type = "negexp")
>
> # linear combination of b + ranef
>
> yi_adj <- sel$b[[1]] + rnorm(sel$k, 0, sqrt(sel$se^2 + sel$vi + sel$tau2))
>
> plot(sims$yi, yi_adj)
> ```
> But of course i'm not sure. Something like using b, tau2, vi and seb to
> estimate the individual effect correcting for publication bias.
> Thank you!
> --
> *Filippo Gambarota, PhD*
> Postdoctoral Researcher - University of Padova
> Department of Developmental and Social Psychology
> Website: filippogambarota.github.io
> Research Groups: Psicostat <https://psicostat.dpss.psy.unipd.it/>