[R-meta] Negative Weights in a Multivariate Meta-Analysis
Prof. Dr. Martin Brunner
m@rt|n@brunner @end|ng |rom un|-pot@d@m@de
Fri Aug 26 11:56:12 CEST 2022
I would like to use a multivariate fixed-effects model to estimate an
average (true) effect size across several dependent effect sizes. The
dependency among effect sizes arises because these effect sizes were
obtained for the same sample of individuals within studies.
To set up my meta-analytic model I use the impute_covariance_matrix function
from the clubsandwich package assuming that the effect sizes correlate with
r = .70 within studies (see below).
The potential problem is now that the average effect size that is estimated
by the fixed-effects model is smaller than any of the observed effect sizes.
This does not seem plausible to me. The reason is that some effect sizes
receive negative weights.
In his great blog post James (Pustejovsky) anticipated such issues (see
Specifically, he wrote: "One wrinkle with the more general form of the
weights is that the effect-size level weights can sometimes be negative
(i.e., negative ). This will tend to happen when the sampling variances
within a study are discrepant, such as when one is much smaller than the
others in study , when the (assumed or estimated) sampling correlation is
high, and when is zero or small. This is something that warrants further
Does anyone have some thougths how one can deal with such situations? For
example, I could imagine using a different weighting scheme where the
within-study sampling variances and covariances are assumed to be the same
(i.e., setting smooth_vi to TRUE). The good thing is that the average effect
size seems now much more plausible to me. However, using this approach
implies that each effect size estimate that was observed for a certain study
receives the same weight which is not well aligned with the idea of using
inverse variance weights.
Any ideas are highly appreciated.
Here is a reprocible R syntax code.
yi <- c(0.27689888, 0.62431603 ,0.09817025, 0.28281538, 0.26482706
vi <- c(0.003103681 ,0.012500222, 0.001569756, 0.003748064, 0.005302781
study <- c(rep(1,3), rep(2,4))
dat <- data.frame(cbind(yi,vi,study))
V_list.1 <- impute_covariance_matrix(vi = dat$vi, cluster = dat$study,r =
.7,smooth_vi = FALSE)
V_list.2 <- impute_covariance_matrix(vi = dat$vi, cluster = dat$study,r =
.7,smooth_vi = TRUE)
res.1 <- rma.mv(yi, V=V_list.1, method="FE", sparse=TRUE, data=dat)
W.1 <- weights(res.1, type="matrix")
sum(rowSums(W.1) * yi) / sum(W.1)
res.2 <- rma.mv(yi, V=V_list.2, method="FE", sparse=TRUE, data=dat)
W.2 <- weights(res.2, type="matrix")
sum(rowSums(W.2) * yi) / sum(W.2)
forest(res.1, slab=c(paste("ES",seq(1:length(yi)))), showweights=TRUE)
forest(res.2, slab=c(paste("ES",seq(1:length(yi)))), showweights=TRUE)
More information about the R-sig-meta-analysis