[R-meta] Mismatch between P-value and 95% CI in rma()
Adelina Artenie
@de||n@@@rten|e @end|ng |rom br|@to|@@c@uk
Wed Aug 17 14:09:16 CEST 2022
Many thanks, Wolfgang. Glad to know my approach is correct. Yes, I use specific units for each predictor variable.
Best,
Adelina
From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Date: Wednesday, 17 August 2022 at 12:47
To: Adelina Artenie <adelina.artenie using bristol.ac.uk>, r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
Subject: RE: [R-meta] Mismatch between P-value and 95% CI in rma()
It is what it is. The calculation appears correct. For example, for the upper CI bound:
exp(coef(my_metar)[2] + qnorm(.975)*sqrt(vcov(my_metar)[2,2]))
Note that the IRR is for a one-unit increase in the predictor variable. Var1 appears to be some year variable and Var2 some kind of percentage. A one year or one percentage point increase doesn't mean much. You could rescale them with mods = ~ I(Var2/10). Then for Var2 you get:
estimate ci.lb ci.ub
I(Var2/10) 0.9395 0.866 1.0193
So, the upper CI corresponds to roughly a 1.02 times (i.e., 2%) increase in the IRR for each 10 percentage points increase in Var2.
Best,
Wolfgang
>-----Original Message-----
>From: Adelina Artenie [mailto:adelina.artenie using bristol.ac.uk]
>Sent: Wednesday, 17 August, 2022 12:08
>To: Viechtbauer, Wolfgang (NP); r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] Mismatch between P-value and 95% CI in rma()
>
>Thanks Wolfgang. I got exactly the same as you. I would have expected the P-value
>to be closer to 0.05 or the upper bound of the 95%CI to be bigger than 1.002
>Adelina
>
>From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Date: Wednesday, 17 August 2022 at 05:35
>To: r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
>Cc: Adelina Artenie <adelina.artenie using bristol.ac.uk>
>Subject: RE: [R-meta] Mismatch between P-value and 95% CI in rma()
>Hi Adelina,
>
>Thanks for providing me with the data to check this. For 'Var2', this is the
>model table:
>
> estimate se zval pval ci.lb ci.ub
>intrcpt 2.7086 0.2171 12.4785 <.0001 2.2832 3.1341
>Var2 -0.0062 0.0042 -1.5013 0.1333 -0.0144 0.0019
>
>And this is the exponentiated coefficient for Var2 with the corresponding CI:
>
> estimate ci.lb ci.ub
>Var2 0.9938 0.9857 1.0019
>
>Note that I rounded the values to 4 decimal places (instead of 2), so now we can
>see that the upper bound is above 1 (as it should be). So there is no mismatch.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Adelina Artenie [mailto:adelina.artenie using bristol.ac.uk]
>>Sent: Tuesday, 16 August, 2022 23:19
>>To: Viechtbauer, Wolfgang (NP)
>>Subject: Re: [R-meta] Mismatch between P-value and 95% CI in rma()
>>
>>ATTACHMENT(S) REMOVED: Ade_dta.xlsx
>>
>>Hi Wolfgang,
>>
>>Thanks for the quick response. I thought I would send you the actual data to
>make
>>sure I get a similar scenario, so just e-mailing you, even if it's anonymised.
>>Please delete once you have looked at it.
>>
>>I attached my code below.
>>Thanks again.
>>Adelina
>>
>>*****
>>*****
>>
>>Ade_dta$Var2<-as.numeric(Ade_dta$Var2, na.strings = "not available")
>>is.numeric(Ade_dta$Var2)
>>
>>my_metar <- rma(measure = "IRLN",
>> xi = cases,
>> ti = prs_yrs_100,
>> add = 0.5,
>> to = "only0",
>> data = Ade_dta,
>> method = "DL",
>> test = "z",
>> slab = author_inci,
>> mods = ~ Var2 ) #Can try Var1 and Var2 here
>>summary(my_metar)
>>round(exp(coef(summary(my_metar))[-1,c("estimate", "ci.lb", "ci.ub")]), 2)
>>
>>*****
>>*****
>>
>>From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>>Date: Tuesday, 16 August 2022 at 21:04
>>To: Adelina Artenie <adelina.artenie using bristol.ac.uk>, r-sig-meta-analysis using r-
>>project.org <r-sig-meta-analysis using r-project.org>
>>Subject: RE: [R-meta] Mismatch between P-value and 95% CI in rma()
>>Hi Adelina,
>>
>>Can you provide a fully reproducible example? Without this, I cannot really tell
>>what is going on.
>>
>>Best,
>>Wolfgang
>>
>>>-----Original Message-----
>>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>>Behalf Of Adelina Artenie
>>>Sent: Tuesday, 16 August, 2022 21:04
>>>To: r-sig-meta-analysis using r-project.org
>>>Subject: [R-meta] Mismatch between P-value and 95% CI in rma()
>>>
>>>Hi everyone,
>>>
>>>I am running a meta-regression analysis for which the outcome is a rate and
>>>several continuous moderator variables. I use rma() in metafor to estimate the
>>>incidence rate ratio (IRR) associated with a 1-unit increase in the moderator.
>>>
>>>If I specify test = "z", my understanding is that both the P-value and the
>95%CI
>>>are estimated using the Wald test (so results should align). However, I see a
>>>small mismatch between the P-value and the 95%CI: the upper bound of the 95% CI
>>>is 1.00 (eg: IRR=0.97; 95%CI: 0.94 1.00), yet the P-value can be anywhere from
>>>P= 0.07 to P= 0.1, depending on the moderator. These P-values seem too high I
>>>would expect them to be closer to 0.05.
>>>
>>>I wonder if I am making a mistake in my code or interpretation, or
>>alternatively,
>>>if a mismatch like this could occur (though Ive never encountered it before..)
>I
>>>wonder also if this can have anything to do with the log transformation.
>>>
>>>I rely on 95%CI when interpreting my results but I like to also present the P-
>>>value, and so Id like to understand what is happening.
>>>
>>>Many thanks for your help.
>>>Best,
>>>Adelina Artenie
>>>
>>>**
>>>Code:
>>>My_rma <- rma(measure = "IRLN",
>>> xi = cases,
>>> ti = prs_yrs_100,
>>> add = 0.5,
>>> to = "only0",
>>> data = Incidence_dat,
>>> method = "DL",
>>> test = "z",
>>> slab = author,
>>> mods = ~ moderator )
>>>summary(My_rma)
>>>round(exp(coef(summary(My_rma))[-1,c("estimate", "ci.lb", "ci.ub")]), 2)
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list