[R-meta] Unrealistic confidence limits for heterogeneity?

[Viechtbauer, Wolfgang (NP)]

Oh, I think you might be right! So here is a dataset to play around with (not a very realistic one):

###############################

dat <- structure(list(yi = c(-0.008, 0.059, 0.029, 0.019, 0.018, 0.036, 0.036,
0.007, -0.032, 0.093), vi = c(0.1538, 0.1111, 0.1379, 0.1212, 0.1538, 0.1333,
0.1212, 0.1111, 0.129, 0.16)), class = "data.frame", row.names = c(NA, -10L))

-min(dat$vi) # -0.1111

res <- rma(yi, vi, data=dat, control=list(tau2.min=-min(dat$vi)))
res

# res$tau2 is -0.1101831 so getting quite close to -min(vi)

# then I essentially did

out <- profile(res, xlim=c(-min(dat$vi), 0.1), steps=100)

# which led me to believe that the likelihood might continue increasing
# if one were to allow a tau^2 value even lower than -min(vi)

# but I forgot that the fit fails with tau2=-min(vi)

head(round(data.frame(tau2=out$tau2, ll=out$ll), 4))

# when we push the lowest value profiled just above -min(vi), we see a huge
# drop in the log likelihood

out <- profile(res, xlim=c(-min(dat$vi)+.000000001, 0.1), steps=100)
head(round(data.frame(tau2=out$tau2, ll=out$ll), 4))

###############################

In fact, one cannot even compute the (log) likelihood for cases where tau^2 is below -min(vi) since this would require taking the square root or log of a negative value (and presumable we don't want to get into complex numbers here ...).

So, it does look like the maximum must indeed be in the interior of (-min(vi), +infinity).

Maybe I should then constrain tau2.min to be >= -min(vi) + something like .Machine$double.eps ...

Best,
Wolfgang

>-----Original Message-----
>From: James Pustejovsky [mailto:jepusto using gmail.com]
>Sent: Monday, 03 April, 2023 0:22
>To: R Special Interest Group for Meta-Analysis
>Cc: Michael Dewey; Viechtbauer, Wolfgang (NP)
>Subject: Re: [R-meta] Unrealistic confidence limits for heterogeneity?
>
><Nerdy follow-up question>
>
>One can of course construct a dataset where the likelihood profile peaks
>essentially at -min(vi) (or rather, just a smidge above it), which might suggest
>that a value of tau^2 that is even smaller than -min(vi) could lead to an even
>larger likelihood, but then one would have to allow negative weights.
>
>Wolfgang, do you know for sure that this is the case? I was under the impression
>that the profile likelihood in tau^2 (and the REML likelihood) should always have
>a maximum in the interior of (-min(vi), +infinity). I haven't worked through a
>full derivation, so was just curious whether you might know to the contrary.