[R-meta] Unrealistic confidence limits for heterogeneity?

[Viechtbauer, Wolfgang (NP)]

Hi James,

Your example made me realize some tricky issues when tau2.min is set to -min(vi) (which is the smallest value of tau2.min allowed, since otherwise vi + tau^2 could become negative) and one then wants to compute CIs for tau^2 (in your example, confint() fails with res2). In the devel version, I now push tau2.min just slightly above -min(vi) in confint() that hopefully deals with this issue (a bit of a hack but seems to work, at least in your example).

res2 <- rma(yi, vi, data=dat, control=list(tau2.min=-min(vi)))
res2
confint(res2)
confint(res2, type="PL")
res2 <- rma(yi, vi, data=dat, method="GENQ", weights=1/vi, control=list(tau2.min=-min(vi)))
confint(res2)

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of James Pustejovsky via R-sig-meta-analysis
>Sent: Thursday, 30 March, 2023 19:00
>To: R Special Interest Group for Meta-Analysis
>Cc: James Pustejovsky
>Subject: Re: [R-meta] Unrealistic confidence limits for heterogeneity?
>
>Some further comments in addition to Michael's response below.
>
>James
>
>1. It is possible to allow for negative heterogeneity estimates using the
>metafor package. Here is an example of the syntax:
>
>library(metafor)
>
># generate data with no heterogeneity
>set.seed(20230330)
>k <- 10
>vi <- 4 / (rpois(k, 22) + 8)
>yi <- rnorm(k, mean = 0.2, sd = sqrt(vi))
>dat <- data.frame(yi, vi)
>
># regular random effects meta-analysis, REML estimator
>res1 <- rma(yi = yi, vi = vi, data=dat)
>res1
>
># allow negative heterogeneity, REML estimator
>res2 <- rma(yi = yi, vi = vi, data=dat, control=list(tau2.min=-min(vi)))
>res2
>
># allow negative heterogeneity, other heterogeneity estimators
>rma(yi = yi, vi = vi, data=dat, method = "ML",
>control=list(tau2.min=-min(vi)))
>rma(yi = yi, vi = vi, data=dat, method = "DL",
>control=list(tau2.min=-min(vi)))
>rma(yi = yi, vi = vi, data=dat, method = "HE",
>control=list(tau2.min=-min(vi)))
>
>2. You can obtain the estimated standard error for tau-squared as follows:
>res1$se.tau2
>res2$se.tau2
>
>3. The metafor package implements several different confidence intervals
>for tau-squared. The GENQ method requires estimating the model with method
>GENQ.
>confint(res1) # confidence interval for tau-squared
>confint(res1, type = "PL") # profile likelihood method
>confint(res1, type = "QP") # Q-profile method
>rma(yi = yi, vi = vi, data=dat, weights = 1 / vi, method = "GENQ")  |>
>  confint(type = "GENQ") # Generalized Q-statistic method