[R-meta] [EXTERNAL] RE: Unusual problem with metafor:rma output

[Simonson, Richard J.]

Thank you, Dr. Viechtbauer, for the clarification. The mechanism behind the phenomenon is definitely more clear to me!


Thank you,

Rich

PhD Candidate


________________________________
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: Friday, July 10, 2020 8:35 AM
To: Simonson, Richard J. <simonsr1 using my.erau.edu>
Cc: r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
Subject: RE: [EXTERNAL] RE: Unusual problem with metafor:rma output

Dear Rich,

Please keep the mailing list in cc.

The estimate of tau^2 may not fall inside of the CI when the method used to estimate tau^2 is not based on the same statistical principle as the method used to construct the CI.

For example, ML/REML estimation of tau^2 is based on a different principle than the Q-profile method that will be used to construct the CI. Usually, this doesn't cause any issues, but can occasionally lead to the estimate falling outside of the CI. Here is an example:

library(metafor)

dat <- structure(list(yi = c(-0.05, -1.86, -0.48, -1.15, -0.57, -0.64,
0.46, -0.53, -0.96, 2.71), vi = c(0.256, 0.928, 0.273, 0.287,
0.249, 0.103, 0.369, 0.061, 0.22, 0.824)), class = "data.frame", row.names = c(NA,
-10L))

res <- rma(yi, vi, data=dat, method="REML")
res
confint(res)

The Q-profile method is more aligned with the PM/EB estimators, so switching to those should solve this issue.

res <- rma(yi, vi, data=dat, method="PM")
res
confint(res)

If you want to stick to ML/REML, then a profile likelihood CI would guarantee that the estimate of tau^2 falls inside of the CI. To obtain such a CI, you can use the rma.mv() function to fit the same model, but then confint() gives you the profile likelihood CI:

dat$id <- 1:10
res <- rma.mv(yi, vi, random = ~ 1 | id, data=dat, method="REML")
res
confint(res)

If you install the 'devel' version of metafor (https://wviechtb.github.io/metafor/#installation), then there is also an undocumented feature of confint() that gives you the profile likelihood CI when the model was fitted with rma():

res <- rma(yi, vi, data=dat, method="REML")
confint(res, type="PL")

Best,
Wolfgang

>-----Original Message-----
>From: Simonson, Richard J. [mailto:simonsr1 using my.erau.edu]
>Sent: Thursday, 09 July, 2020 20:03
>To: Viechtbauer, Wolfgang (SP)
>Subject: Re: [EXTERNAL] RE: Unusual problem with metafor:rma output
>
>Dr. Viechtbauer,
>
>Thank you for the quick response! Reading through the help section was
>helpful. I'm having difficulty, however, in conceptualizing how to describe
>this phenomenon to my audience. Specifically, where the use of
>the DerSimonian and Laird methods is the only method that gives results in
>the estimate/C.I boundary break.
>
>Would you be able to point me in the direction of a literature source that
>may describe what may be happening here? I rerfer particularly to the
>subsection of the help text:
>
>"Depending on the method used to estimate � and the width of the confidence
>interval, it can happen that the confidence interval does not actually
>contain the estimate (trying to explain this to reviewers can be tricky"
>
>Thank you for your time!
>
>Thank you,
>Rich
>PhD Candidate
>
>________________________________________
>From: Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Thursday, July 9, 2020 9:47 AM
>To: Simonson, Richard J. <simonsr1 using my.erau.edu>; r-sig-meta-analysis using r-
>project.org <r-sig-meta-analysis using r-project.org>
>Subject: [EXTERNAL] RE: Unusual problem with metafor:rma output
>
>Dear Rich,
>
>See help(confint.rma.uni) and see the 'Details' section.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-
>project.org]
>>On Behalf Of Simonson, Richard J.
>>Sent: Thursday, 09 July, 2020 15:36
>>To: r-sig-meta-analysis using r-project.org
>>Subject: [R-meta] Unusual problem with metafor:rma output
>>
>>I have a rather unusual problem with the output of my metaregression using
>>metafor:rma.
>>
>>When I attempt to calculate the confidence intervals for the tau, I, and H
>>values, all of the estimates are lower than the lower 95% confidence
>>interval:
>>
>>       estimate   ci.lb   ci.ub
>>tau^2    0.8002  1.8334  7.8651
>>tau      0.9125  1.456  2.7222
>>I^2(%)  92.455  95.157 99.3356
>>H^2     11.5454 24.2385 90.6577
>>
>>The code I used for the random effects meta-regression is:
>>
>>res <- rma(yi = LogOdds, sei = SE, data = data, method = 'DL')
>>
>>After playing around with the code, I discovered that if I change the
>method
>>to method = "SJ", this problem is eliminated but increases all
>heterogeneity
>>calculations significantly.
>>
>>I'm guessing this has to do with how these parameters are being estimated.
>>But why was this happening in the first place?
>>
>>Has anyone encountered this before?
>>
>>Thank you,
>>
>>Rich
>>
>>PhD Candidate

	[[alternative HTML version deleted]]