[R-meta] Negative Weights in a Multivariate Meta-Analysis

[Prof. Dr. Martin Brunner]

Dear listmembers,
I would like to use a multivariate fixed-effects model to estimate an 
average (true) effect size across several dependent effect sizes. The 
dependency among effect sizes arises because these effect sizes were 
obtained for the same sample of individuals within studies.
To set up my meta-analytic model I use the impute_covariance_matrix function 
from the clubsandwich package assuming that the effect sizes correlate with 
r = .70 within studies (see below).
The potential problem is now that the average effect size that is estimated 
by the fixed-effects model is smaller than any of the observed effect sizes. 
This does not seem plausible to me. The reason is that some effect sizes 
receive negative weights.
In his great blog post James (Pustejovsky) anticipated such issues (see 
https://www.jepusto.com/weighting-in-multivariate-meta-analysis/): 
Specifically, he wrote: "One wrinkle with the more general form of the 
weights is that the effect-size level weights can sometimes be negative 
(i.e., negative ). This will tend to happen when the sampling variances 
within a study are discrepant, such as when one is much smaller than the 
others in study , when the (assumed or estimated) sampling correlation is 
high, and when is zero or small. This is something that warrants further 
investigation."
Does anyone have some thougths how one can deal with such situations? For 
example, I could imagine using a different weighting scheme where the 
within-study sampling variances and covariances are assumed to be the same 
(i.e., setting smooth_vi to TRUE). The good thing is that the average effect 
size seems now much more plausible to me. However, using this approach 
implies that each effect size estimate that was observed for a certain study 
receives the same weight which is not well aligned with the idea of using 
inverse variance weights.
Any ideas are highly appreciated.
Best,
Martin


Here is a reprocible R syntax code.


library(clubSandwich)
library(metafor)

yi <- c(0.27689888, 0.62431603 ,0.09817025, 0.28281538, 0.26482706 
,0.17486036, 0.15944605)
vi <- c(0.003103681 ,0.012500222, 0.001569756, 0.003748064, 0.005302781 
,0.002147452, 0.004888748)
study <- c(rep(1,3), rep(2,4))



dat <- data.frame(cbind(yi,vi,study))


V_list.1 <- impute_covariance_matrix(vi = dat$vi, cluster = dat$study,r = 
.7,smooth_vi = FALSE)
V_list.1

V_list.2 <- impute_covariance_matrix(vi = dat$vi, cluster = dat$study,r = 
.7,smooth_vi = TRUE)
V_list.2


res.1 <- rma.mv(yi, V=V_list.1, method="FE", sparse=TRUE, data=dat)
res.1

W.1 <- weights(res.1, type="matrix")
rowSums(W.1)
sum(rowSums(W.1) * yi) / sum(W.1)


res.2 <- rma.mv(yi, V=V_list.2, method="FE", sparse=TRUE, data=dat)
res.2

W.2 <- weights(res.2, type="matrix")
rowSums(W.2)
sum(rowSums(W.2) * yi) / sum(W.2)



par(mfrow=c(1,2))

forest(res.1, slab=c(paste("ES",seq(1:length(yi)))), showweights=TRUE)
title("rho=0.7, smooth_vi=FALSE")

forest(res.2, slab=c(paste("ES",seq(1:length(yi)))), showweights=TRUE)
title("rho=0.7, smooth_vi=TRUE")