[R-meta] Mismatch between P-value and 95% CI in rma()

[Viechtbauer, Wolfgang (NP)]

Hi Adelina,

Thanks for providing me with the data to check this. For 'Var2', this is the model table:

         estimate      se     zval    pval    ci.lb   ci.ub      
intrcpt    2.7086  0.2171  12.4785  <.0001   2.2832  3.1341
Var2      -0.0062  0.0042  -1.5013  0.1333  -0.0144  0.0019

And this is the exponentiated coefficient for Var2 with the corresponding CI:

     estimate  ci.lb  ci.ub
Var2   0.9938 0.9857 1.0019

Note that I rounded the values to 4 decimal places (instead of 2), so now we can see that the upper bound is above 1 (as it should be). So there is no mismatch.

Best,
Wolfgang

>-----Original Message-----
>From: Adelina Artenie [mailto:adelina.artenie using bristol.ac.uk]
>Sent: Tuesday, 16 August, 2022 23:19
>To: Viechtbauer, Wolfgang (NP)
>Subject: Re: [R-meta] Mismatch between P-value and 95% CI in rma()
>
>ATTACHMENT(S) REMOVED: Ade_dta.xlsx
>
>Hi Wolfgang,
>
>Thanks for the quick response. I thought I would send you the actual data to make
>sure I get a similar scenario, so just e-mailing you, even if it’s anonymised.
>Please delete once you have looked at it.
>
>I attached my code below.
>Thanks again.
>Adelina
>
>*****
>*****
>
>Ade_dta$Var2<-as.numeric(Ade_dta$Var2, na.strings = "not available")
>is.numeric(Ade_dta$Var2)
>
>my_metar <- rma(measure = "IRLN",
>                    xi = cases,
>                    ti = prs_yrs_100,
>                    add = 0.5,
>                    to = "only0",
>                    data = Ade_dta,
>                    method = "DL",
>                    test = "z",
>                    slab = author_inci,
>                    mods = ~ Var2 ) #Can try Var1 and Var2 here
>summary(my_metar)
>round(exp(coef(summary(my_metar))[-1,c("estimate", "ci.lb", "ci.ub")]), 2)
>
>*****
>*****
>
>From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Date: Tuesday, 16 August 2022 at 21:04
>To: Adelina Artenie <adelina.artenie using bristol.ac.uk>, r-sig-meta-analysis using r-
>project.org <r-sig-meta-analysis using r-project.org>
>Subject: RE: [R-meta] Mismatch between P-value and 95% CI in rma()
>Hi Adelina,
>
>Can you provide a fully reproducible example? Without this, I cannot really tell
>what is going on.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>Behalf Of Adelina Artenie
>>Sent: Tuesday, 16 August, 2022 21:04
>>To: r-sig-meta-analysis using r-project.org
>>Subject: [R-meta] Mismatch between P-value and 95% CI in rma()
>>
>>Hi everyone,
>>
>>I am running a meta-regression analysis for which the outcome is a rate and
>>several continuous moderator variables. I use rma() in metafor to estimate the
>>incidence rate ratio (IRR) associated with a 1-unit increase in the moderator.
>>
>>If I specify test = "z", my understanding is that both the P-value and the 95%CI
>>are estimated using the Wald test (so results should align). However, I see a
>>small mismatch between the P-value and the 95%CI: the upper bound of the 95% CI
>>is 1.00 (eg: IRR=0.97; 95%CI: 0.94  1.00), yet the P-value can be anywhere from
>>P= 0.07 to P= 0.1, depending on the moderator. These P-values seem too high  I
>>would expect them to be closer to 0.05.
>>
>>I wonder if I am making a mistake in my code or interpretation, or
>alternatively,
>>if a mismatch like this could occur (though Ive never encountered it before..) I
>>wonder also if this can have anything to do with the log transformation.
>>
>>I rely on 95%CI when interpreting my results but I like to also present the P-
>>value, and so Id like to understand what is happening.
>>
>>Many thanks for your help.
>>Best,
>>Adelina Artenie
>>
>>**
>>Code:
>>My_rma <- rma(measure = "IRLN",
>>                        xi = cases,
>>                        ti = prs_yrs_100,
>>                        add = 0.5,
>>                        to = "only0",
>>                        data = Incidence_dat,
>>                        method = "DL",
>>                        test = "z",
>>                        slab = author,
>>                        mods = ~ moderator )
>>summary(My_rma)
>>round(exp(coef(summary(My_rma))[-1,c("estimate", "ci.lb", "ci.ub")]), 2)