[R-meta] Unrealistic confidence limits for heterogeneity?

[Michael Dewey]

This raises the question of what to do when an estimate is at the 
boundary of the parameter space (or beyond it). Is it worth warning the 
user that this may have happened (ie the lower bound is at -min(vi)? It 
suggests to me that the user has here an attempt to fit a model to a 
dataset which is inadequate for that model.

Michael

On 01/04/2023 13:17, Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis 
wrote:
> Hi James,
> 
> Your example made me realize some tricky issues when tau2.min is set to -min(vi) (which is the smallest value of tau2.min allowed, since otherwise vi + tau^2 could become negative) and one then wants to compute CIs for tau^2 (in your example, confint() fails with res2). In the devel version, I now push tau2.min just slightly above -min(vi) in confint() that hopefully deals with this issue (a bit of a hack but seems to work, at least in your example).
> 
> res2 <- rma(yi, vi, data=dat, control=list(tau2.min=-min(vi)))
> res2
> confint(res2)
> confint(res2, type="PL")
> res2 <- rma(yi, vi, data=dat, method="GENQ", weights=1/vi, control=list(tau2.min=-min(vi)))
> confint(res2)
> 
> Best,
> Wolfgang
> 
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>> Behalf Of James Pustejovsky via R-sig-meta-analysis
>> Sent: Thursday, 30 March, 2023 19:00
>> To: R Special Interest Group for Meta-Analysis
>> Cc: James Pustejovsky
>> Subject: Re: [R-meta] Unrealistic confidence limits for heterogeneity?
>>
>> Some further comments in addition to Michael's response below.
>>
>> James
>>
>> 1. It is possible to allow for negative heterogeneity estimates using the
>> metafor package. Here is an example of the syntax:
>>
>> library(metafor)
>>
>> # generate data with no heterogeneity
>> set.seed(20230330)
>> k <- 10
>> vi <- 4 / (rpois(k, 22) + 8)
>> yi <- rnorm(k, mean = 0.2, sd = sqrt(vi))
>> dat <- data.frame(yi, vi)
>>
>> # regular random effects meta-analysis, REML estimator
>> res1 <- rma(yi = yi, vi = vi, data=dat)
>> res1
>>
>> # allow negative heterogeneity, REML estimator
>> res2 <- rma(yi = yi, vi = vi, data=dat, control=list(tau2.min=-min(vi)))
>> res2
>>
>> # allow negative heterogeneity, other heterogeneity estimators
>> rma(yi = yi, vi = vi, data=dat, method = "ML",
>> control=list(tau2.min=-min(vi)))
>> rma(yi = yi, vi = vi, data=dat, method = "DL",
>> control=list(tau2.min=-min(vi)))
>> rma(yi = yi, vi = vi, data=dat, method = "HE",
>> control=list(tau2.min=-min(vi)))
>>
>> 2. You can obtain the estimated standard error for tau-squared as follows:
>> res1$se.tau2
>> res2$se.tau2
>>
>> 3. The metafor package implements several different confidence intervals
>> for tau-squared. The GENQ method requires estimating the model with method
>> GENQ.
>> confint(res1) # confidence interval for tau-squared
>> confint(res1, type = "PL") # profile likelihood method
>> confint(res1, type = "QP") # Q-profile method
>> rma(yi = yi, vi = vi, data=dat, weights = 1 / vi, method = "GENQ")  |>
>>   confint(type = "GENQ") # Generalized Q-statistic method
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 

-- 
Michael
http://www.dewey.myzen.co.uk/home.html