[R-meta] [metafor] Mantel-Haenszel method - rma.mh vs meta(metabin)

[Viechtbauer, Wolfgang (NP)]

Dear Marc,

You are using different variables as input to metabin() and later on (the dataset you attached also doesn't include the variables that you use in metabin()).

But even if you use the same variables, you won't get the same results. metabin() provides a random-effects version of the MH method that I do not provide in the metafor package.

The method comes from RevMan, which (a) uses the estimate from the MH method for estimating the amount of heterogeneity and (b) then switches to an inverse-variance method. To me, this always seemed a bit strange and so I did not implement this in metafor.

To illustrate:

m.sln <- metabin(event.e = ICG_pos,
                 n.e = ICG_total,
                 event.c = RI_pos,
                 n.c = RI_total,
                 studlab = Study,
                 data = df,
                 sm = "RD",
                 method = "MH",
                 MH.exact = TRUE,
                 fixed = FALSE,
                 random = TRUE,
                 method.tau = "PM",
                 hakn = TRUE,
                 title = "SLN detection rate")
m.sln

df <- escalc(measure = "RD", data=df, addyi=FALSE,
              ai = ICG_pos, ci = RI_pos,
              n1i = ICG_total, n2i = RI_total,
              slab = paste(Study, Year, sep = ", "))
rma(df, test="knha", tau2=m.sln$tau2)

So when I grab the tau^2 estimate from metabin() and fit a standard random-effects model (using the inverse-variance method), I can get the same results. There is one other small adjustment: In meta, when using measure RD, the +0.5 adjustment is only applied when calculating the sampling variances, not the estimates themselves, and I can force the same behavior in escalc() by setting addyi=FALSE.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of marc.harms using mh-analytics.eu
>Sent: Monday, 30 January, 2023 20:31
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] [metafor] Mantel-Haenszel method - rma.mh vs meta(metabin)
>
>ATTACHMENT(S) REMOVED: Breast-Cancer-SLN-detection-RD-Jan2023-2.xlsx
>
>Hi all,
>
>I have the following scenario: 2x2 table, binary data
>
>Multiple studies are comparing the SLN detection in breast cancer patients of two
>different detection methods (ICG vs RI).
>The literature review is providing 13 studies, table is attached.
>
>Aim is to calculate the risk difference. Since the detection rate is a binary
>outcome, the inverse variance method might not be a good choice for calculating
>the weights. Therefore, I wanted to use the Mantel-Haenszel method.
>
>When I am using the metabin function from [meta], I can calculate the pooled
>effect as follows:
>
>m.sln<- metabin(event.e = ICG_SLN,
>                 n.e = ICG_total,
>                 event.c = RI_SLN,
>                 n.c = RI_total,
>                 studlab = Study,
>                 data = df,
>                 sm = "RD",
>                 method = "MH",
>                 MH.exact = TRUE,
>                 fixed = FALSE,
>                 random = TRUE,
>                 method.tau = "PM",
>                 hakn = TRUE,
>                 title = "SLN detection rate")
>
>My problem: I am not able to run this utilizing the rma.mh function in [metafor]
>for RD. The outcomes are different:
>
>[META]
>                         RD            95%-CI    t p-value
>Random effects model 0.0112 [-0.0224; 0.0447] 0.73  0.4821
>
>Quantifying heterogeneity:
>tau^2 = 0.0021 [0.0004; 0.0092]; tau = 0.0455 [0.0197; 0.0960]
>I^2 = 63.2% [33.0%; 79.7%]; H = 1.65 [1.22; 2.22]
>
>Test of heterogeneity:
>     Q d.f. p-value
>32.57   12  0.0011
>
>dat <- escalc(measure = "RD",
>              ai = ICG_pos, ci = RI_pos,
>              n1i = ICG_total, n2i = RI_total,
>              slab = paste(Study, Year, sep = ", "),
>
>                                              )
>res <- rma(dat, method="REML")
>[METAFOR] - rma, random effects
>Random-Effects Model (k = 13; tau^2 estimator: REML)
>
>  logLik  deviance       AIC       BIC      AICc
> 17.5702  -35.1404  -31.1404  -30.1705  -29.8070
>
>tau^2 (estimated amount of total heterogeneity): 0.0013 (SE = 0.0008)
>tau (square root of estimated tau^2 value):      0.0355
>I^2 (total heterogeneity / total variability):   73.23%
>H^2 (total variability / sampling variability):  3.74
>
>Test for Heterogeneity:
>Q(df = 12) = 31.9726, p-val = 0.0014
>
>Model Results:
>
>estimate      se    zval    pval    ci.lb   ci.ub
>  0.0111  0.0129  0.8631  0.3881  -0.0141  0.0364
>
>res_1 <- rma.mh(measure = "RD",
>                ai = ICG_pos, ci = RI_pos,
>                n1i = ICG_total, n2i = RI_total, data=dat)
>[METAFOR] rma.mh
>Equal-Effects Model (k = 13)
>
>  logLik  deviance       AIC       BIC      AICc
> 18.3474   32.0734  -34.6948  -34.1299  -34.3312
>
>I^2 (total heterogeneity / total variability):  62.59%
>H^2 (total variability / sampling variability): 2.67
>
>Test for Heterogeneity:
>Q(df = 12) = 32.0734, p-val = 0.0013
>
>Model Results:
>
>estimate      se    zval    pval   ci.lb   ci.ub
>  0.0119  0.0058  2.0411  0.0412  0.0005  0.0234
>
>hanks, Marc