[R-meta] Problems using rma.mh

[Viechtbauer, Wolfgang (SP)]

Hi David, Hi Gerta,

Thanks for the note, David. This was an oversight in the printing function. When there are no events at all in one of the arms across all studies, then certain statistics cannot be computed, which are then NA. This leads to problems when trying to format these values with sprintf(). I'll fix this asap.

But rma.mh() itself works, so this is fine:

dat <- dat.nielweise2007
res <- rma.mh(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i, data=dat[which(dat$ai==0),])

If you are interested in the Breslow-Day and Tarone statistics, you can still get them from the object with:

res$BD
res$TA

and the corresponding p-values with:

res$BDp
res$TAp

But in this case, you will find that both statistics are actually NA. In other words, these tests cannot be conducted when there are no events in one of the two arms. Or one would have to use continuity corrections, but the point of the MH method and related statistics (such as BD and TA) is that they do not require such corrections and in fact were never meant to be combined with such corrections.

Interestingly, the Cochran-Mantel-Haenszel test can still be conducted:

res$CO
res$COp

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
>On Behalf Of David Fisher
>Sent: Friday, 12 June, 2020 12:50
>To: Gerta Ruecker; wvb using wvbauer.com
>Cc: r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] Problems using rma.mh
>
>Dear Gerta,
>
>Thanks -- it's good at least to know that it's not only me seeing the error
>:-)
>
>Regarding continuity correction: I agree;  but given that a correction
>is sometimes applied even to Mantel-Haenszel pooling in extreme cases
>(such as when all studies have zero events in the same arm, as in my
>example), I wondered whether a similar argument could/should be
>applied to other associated statistics.
>
>I have copied Wolfgang into this email:  Wolfgang, could you take a
>look at the issue with rma.mh, please?
>
>Many thanks,
>
>David.
>
>On Fri, Jun 12, 2020 at 10:54 AM Gerta Ruecker
><ruecker using imbi.uni-freiburg.de> wrote:
>>
>> Dear David,
>>
>> With "I can't locate" I meant that I see the same error, but don't know
>> why :-(
>>
>> With respect to the homogeneity statistics you asked for: I don't think
>> they are implemented, this would be a question for Guido (who will be
>> back at work on Monday). I don't find any hint to them when typing
>> help(metabin).
>>
>> In principle, I think it preferable to avoid continuity corrections.
>>
>> Maybe Wolfgang can solve the problem with rma.mh and also answer the
>> other question?
>>
>> Best,
>>
>> Gerta
>>
>> Am 12.06.2020 um 11:45 schrieb David Fisher:
>> > Dear Gerta,
>> > Many thanks for your reply.
>> >
>> > Firstly: apologies, I was inconsistent in the naming of my dataset in
>> > my example code, as you spotted.  But no, this doesn't solve my
>> > original problem :-)
>> > When you say "I cannot locate", do you mean you do not see the same
>> > error message?
>> >
>> > Thankyou for pointing me towards the alternative package "metabin".
>> > Can you tell me whether it can be used to calculate the Breslow-Day
>> > (and/or Tarone) statistics for homogeneity of odds ratios?   Or, could
>> > you tell me whether you would expect these statistics to be calculated
>> > using raw counts, or using continuity-corrected counts?
>> >
>> > Thanks again,
>> > David.
>> >
>> > On Fri, Jun 12, 2020 at 10:24 AM Gerta Ruecker
>> > <ruecker using imbi.uni-freiburg.de> wrote:
>> >> Once more. Here is the code with OR and without continuity correction:
>> >>
>> >> library(metafor)
>> >> dat2007 <- dat.nielweise2007
>> >> library(meta)
>> >> m1 <- metabin(ai, n1i, ci, n2i, sm = "OR", incr = 0, data = dat2007)
>> >> m2 <- metabin(ai, n1i, ci, n2i, sm = "OR", incr = 0,
>> >> data=dat2007[dat2007$ai==0,])
>> >> m1
>> >> m2
>> >>
>> >> For m2, with only zero events, you obtain OR = 0, but without estimates
>> >> for the standard error and thus without confidence intervals. See
>> >>
>> >> forest(m2)
>> >>
>> >> Best,
>> >>
>> >> Gerta
>> >>
>> >> Am 12.06.2020 um 11:15 schrieb Gerta Ruecker:
>> >>> Dear David,
>> >>>
>> >>> Your code defines a data set called dat, but not dat2007. But this
>> >>> doesn't seem to be the problem (I can't locate). Maybe the double zero
>> >>> is the problem?
>> >>>
>> >>> You could try the meta package:
>> >>>
>> >>> library(metafor)
>> >>> dat2007 <- dat.nielweise2007
>> >>> library(meta)
>> >>> m1 <- metabin(ai, n1i, ci, n2i, data = dat2007)
>> >>> m2 <- metabin(ai, n1i, ci, n2i, data=dat2007[dat2007$ai==0,])
>> >>> m1
>> >>> m2
>> >>>
>> >>> Best,
>> >>>
>> >>> Gerta
>> >>>
>> >>> Am 12.06.2020 um 11:01 schrieb David Fisher:
>> >>>> Dear metafor community,
>> >>>>
>> >>>> I am a Stata user with only limited knowledge of R.  I am trying to
>> >>>> validate some Stata code by comparing with output from metafor; in
>> >>>> particular, sparse data/zero cells using Mantel-Haenszel methods.
>> >>>>
>> >>>> My reference is Wolfgang's page
>> >>>> "http://www.metafor-
>project.org/doku.php/tips:comp_mh_different_software".
>> >>>>
>> >>>> If I type:
>> >>>>
>> >>>> library(metafor)
>> >>>> dat <- dat.nielweise2007
>> >>>> rma.mh(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i, data=dat)
>> >>>>
>> >>>> ...then I successfully obtain the output shown on the webpage.
>> >>>> However, if I alter the command in (seemingly) any way, I get the
>> >>>> following error:
>> >>>>
>> >>>> Error in sprintf(format, names(x)) :
>> >>>>     invalid format '%NA'; use format %s for character objects
>> >>>>
>> >>>> For instance, if I limit the dataset to studies with zero events in
>> >>>> the treatment arm:
>> >>>>
>> >>>> rma.mh(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i,
>> >>>> data=dat2007[which(dat2007$ai==0),])
>> >>>>
>> >>>> ...and similarly if I attempt to use another data frame, or import
>> >>>> from Stata using "read.dta".  Furthermore, I do not appear to have
>the
>> >>>> same issue if I use another model.  For instance, the following runs
>> >>>> with no problem:
>> >>>>
>> >>>> rma(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i,
>> >>>> data=dat2007[which(dat2007$ai==0),])
>> >>>>
>> >>>> I am using a fresh install of R 4.0.1 and of the metafor package.
>> >>>>
>> >>>> Could someone point me in the right direction?
>> >>>>
>> >>>> Many thanks,
>> >>>>
>> >>>> David.