[R-meta] metafor::rma-function: Statistically significant interaction, but increased tau2 – and how to get the slope from the output

Sabrina Mai Nielsen @@br|n@@m@|@n|e|@en @end|ng |rom reg|onh@dk
Thu Aug 1 20:52:21 CEST 2019

I am having troubles making sense of a REML-based meta-regression with an interaction between a factor and numeric variable, fitted with rma.

In my study, I have included RCTs from 19 meta-analyses to investigate the association between %women in the RCTs and the effect sizes (i.e. logOR).
The slope for %women, without interaction, is not significant (see below, M_REML0, p=0.6858), and I am now investigating the interaction between %women (CF1_Women..IMP) and the ID for the meta-analysis (i.e. if the slope for %women varies in the different meta-analyses).

1) The interaction seem significant when testing for moderators, but the between-trial variation, tau^2, does not decrease as expected. Why is that?
(When I try method="ML" instead of REML, then tau^2 does decrease as expected; see anova(M0,M1) below)

M_REML0 (without interaction), tau^2 = 0.2076
M_REML1 (with an interaction), tau^2 = 0.2127, Test of Moderators (interaction terms) p-val = 0.0044
(see full outputs below)

2) I am in doubt how to get the estimate for the slopes – e.g. for meta-analysis with ID 8.

estimate      se     zval    pval     ci.lb    ci.ub
intrcpt               -0.3567  1.0739  -0.3322  0.7398   -2.4616   1.7481
CF1_Women..IMP         0.0114  0.0139   0.8244  0.4097   -0.0157   0.0386
(...)
CF1_Women..IMP:id8    -0.0718  0.0214  -3.3572  0.0008   -0.1137  -0.0299  ***
(see full output below)

I would think it would be calculated as CF1_Women..IMP + CF1_Women..IMP:id8, i.e. 0.0114 +(-0.0718) = -0.0604 (with corresponding 95%CI of -0.1294 to 0.0087). Is that correct?
Or is the slope for ID 8 simply the estimate reported for CF1_Women..IMP:id8, i.e. (-0.0718, 95%CI -0.1137 to -0.0299)?
I am asking because when I do a meta-regression with only RCTs for ID 8, I get -0.0940 (95%CI -0.1864 to -0.0016; output not shown), which is more similar to the CF1_Women..IMP:id8 estimate.

Also, when I calculate the slopes for all the meta-analysis IDs, none of them are significantly different from 0, despite the interaction term is significant. I guess that is possible, but it makes me doubt the way I calculated the slopes.

Best,
Sabrina

=======A FEW RELEVANT OUTPUTS=======
> M_REML0<-rma(logOR, logOR.var, mods= ~CF1_Women..IMP + id, method="REML", data=d, btt=2)
> M_REML1<-rma(logOR, logOR.var, mods= ~CF1_Women..IMP * id, method="REML", data=d, btt=c(21:38))
> M_REML0

Mixed-Effects Model (k = 212; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):     0.2076 (SE = 0.0357)
tau (square root of estimated tau^2 value):             0.4556
I^2 (residual heterogeneity / unaccounted variability): 66.60%
H^2 (unaccounted variability / sampling variability):   2.99
R^2 (amount of heterogeneity accounted for):            63.70%

Test for Residual Heterogeneity:
QE(df = 192) = 545.9074, p-val < .0001

Test of Moderators (coefficient 2):
QM(df = 1) = 0.1637, p-val = 0.6858

Model Results:

estimate      se     zval    pval    ci.lb    ci.ub
intrcpt           0.4156  0.3105   1.3383  0.1808  -0.1931   1.0242
CF1_Women..IMP    0.0013  0.0032   0.4046  0.6858  -0.0049   0.0075
id2               0.0672  0.3965   0.1695  0.8654  -0.7099   0.8443
id3               0.5059  0.2521   2.0066  0.0448   0.0117   1.0000    *
id4               0.1753  0.2739   0.6402  0.5221  -0.3615   0.7121
id5               0.2610  0.2558   1.0201  0.3077  -0.2404   0.7624
id6               0.0351  0.2763   0.1272  0.8988  -0.5065   0.5768
id7               0.8416  0.3464   2.4295  0.0151   0.1626   1.5205    *
id8               0.1775  0.2723   0.6519  0.5145  -0.3562   0.7112
id9              -0.3137  0.2684  -1.1690  0.2424  -0.8398   0.2123
id10             -0.8378  0.2723  -3.0765  0.0021  -1.3715  -0.3040   **
id11              0.2878  0.2106   1.3665  0.1718  -0.1250   0.7006
id12              0.1932  0.3449   0.5603  0.5753  -0.4827   0.8692
id13              0.1538  0.2959   0.5199  0.6031  -0.4261   0.7337
id14             -1.0020  0.3432  -2.9195  0.0035  -1.6747  -0.3293   **
id15             -0.6162  0.2739  -2.2497  0.0245  -1.1531  -0.0794    *
id16             -1.5704  0.3394  -4.6269  <.0001  -2.2356  -0.9052  ***
id17             -1.4576  0.2942  -4.9552  <.0001  -2.0341  -0.8811  ***
id18             -0.0457  0.4967  -0.0921  0.9266  -1.0193   0.9278
id19             -0.3440  0.3283  -1.0479  0.2947  -0.9874   0.2994

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

> M_REML1

Mixed-Effects Model (k = 212; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):     0.2127 (SE = 0.0378)
tau (square root of estimated tau^2 value):             0.4612
I^2 (residual heterogeneity / unaccounted variability): 67.50%
H^2 (unaccounted variability / sampling variability):   3.08
R^2 (amount of heterogeneity accounted for):            62.80%

Test for Residual Heterogeneity:
QE(df = 174) = 488.8555, p-val < .0001

Test of Moderators (coefficients 21:38):
QM(df = 18) = 37.5693, p-val = 0.0044

Model Results:

estimate      se     zval    pval     ci.lb    ci.ub
intrcpt               -0.3567  1.0739  -0.3322  0.7398   -2.4616   1.7481
CF1_Women..IMP         0.0114  0.0139   0.8244  0.4097   -0.0157   0.0386
id2                    8.6017  8.6515   0.9942  0.3201   -8.3549  25.5583
id3                    0.7355  1.3494   0.5451  0.5857   -1.9093   3.3804
id4                    0.2098  1.2889   0.1628  0.8707   -2.3164   2.7360
id5                    1.7079  1.1849   1.4414  0.1495   -0.6144   4.0302
id6                   -3.6331  2.2106  -1.6435  0.1003   -7.9658   0.6995
id7                   -5.4865  2.7547  -1.9917  0.0464  -10.8857  -0.0874    *
id8                    5.2745  1.5782   3.3421  0.0008    2.1812   8.3677  ***
id9                    0.4174  1.2215   0.3417  0.7326   -1.9766   2.8114
id10                  -0.1295  1.2015  -0.1078  0.9142   -2.4844   2.2253
id11                   0.2269  1.7098   0.1327  0.8944   -3.1242   3.5779
id12                  -0.7519  2.3333  -0.3222  0.7473   -5.3250   3.8212
id13                  -0.0701  3.4723  -0.0202  0.9839   -6.8758   6.7355
id14                  -0.1842  2.0870  -0.0882  0.9297   -4.2747   3.9063
id15                   0.9002  1.3974   0.6442  0.5194   -1.8386   3.6389
id16                   6.4300  7.6265   0.8431  0.3992   -8.5176  21.3776
id17                   0.7142  2.1172   0.3373  0.7359   -3.4353   4.8638
id18                  -0.4641  1.5171  -0.3059  0.7597   -3.4376   2.5095
id19                   1.2194  4.9584   0.2459  0.8057   -8.4989  10.9376
CF1_Women..IMP:id2    -0.1243  0.1269  -0.9795  0.3274   -0.3730   0.1244
CF1_Women..IMP:id3    -0.0026  0.0177  -0.1480  0.8824   -0.0374   0.0322
CF1_Women..IMP:id4     0.0012  0.0174   0.0672  0.9464   -0.0329   0.0353
CF1_Women..IMP:id5    -0.0207  0.0157  -1.3162  0.1881   -0.0515   0.0101
CF1_Women..IMP:id6     0.0533  0.0308   1.7332  0.0831   -0.0070   0.1136    .
CF1_Women..IMP:id7     0.0690  0.0313   2.2031  0.0276    0.0076   0.1304    *
CF1_Women..IMP:id8    -0.0718  0.0214  -3.3572  0.0008   -0.1137  -0.0299  ***
CF1_Women..IMP:id9    -0.0095  0.0162  -0.5858  0.5580   -0.0413   0.0223
CF1_Women..IMP:id10   -0.0092  0.0156  -0.5907  0.5547   -0.0399   0.0214
CF1_Women..IMP:id11    0.0004  0.0217   0.0167  0.9867   -0.0422   0.0430
CF1_Women..IMP:id12    0.0112  0.0290   0.3852  0.7001   -0.0456   0.0679
CF1_Women..IMP:id13    0.0042  0.0496   0.0856  0.9318   -0.0929   0.1014
CF1_Women..IMP:id14   -0.0108  0.0281  -0.3837  0.7012   -0.0658   0.0443
CF1_Women..IMP:id15   -0.0207  0.0186  -1.1133  0.2656   -0.0570   0.0157
CF1_Women..IMP:id16   -0.0834  0.0777  -1.0733  0.2831   -0.2356   0.0689
CF1_Women..IMP:id17   -0.0252  0.0239  -1.0543  0.2918   -0.0721   0.0217
CF1_Women..IMP:id18    0.0186  0.0274   0.6814  0.4956   -0.0350   0.0723
CF1_Women..IMP:id19   -0.0203  0.0637  -0.3189  0.7498   -0.1452   0.1046

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

>
> M0<-rma(logOR, logOR.var, mods= ~CF1_Women..IMP + id, method="ML", data=d)
> M1<-rma(logOR, logOR.var, mods= ~CF1_Women..IMP * id, method="ML", data=d, btt=c(21:38))
> anova(M0,M1)

df      AIC      BIC     AICc    logLik     LRT   pval       QE  tau^2    R^2
Full    39 515.2691 646.1760 533.4086 -218.6346                488.8555 0.1182
Reduced 21 520.4230 590.9113 525.2862 -239.2115 41.1539 0.0014 545.9074 0.1510 21.75%