[R-sig-ME] metafor - Appropriate heterogeneity estimator when heterogeneity is extreme

[Theodore Lytras]

Hello everyone,

Sorry if this is not the right forum.
I am undertaking a meta-regression using the metafor package, with six binary 
predictors and no intercept (I am only interested in the coefficients, not in 
the pooled effect). I have a sufficient number of studies (N=52), and the 
study effects (log Risk Ratios) show substantial left skew when plotted on a 
normal plot.

The problem is that I get very different estimates for tau^2 (and therefore 
quite different SEs for the regression coefficients) depending on whether I 
use the DerSimonian-Laird (DL) estimator or the REML estimator. 
In both cases heterogeneity is extreme (I^2>99, H^2>150), but t^2 is twice as 
big with REML (0.1901) than with DL (0.0805).

Could someone give me any clues as to which estimator may be more appropriate 
in such a situation? 
Also, what may be the cause of such greatly divergent estimates? Is it the 
heterogeneity? The non-normal distribution of study effects? Or something 
else?

Here's the output:

> rma.uni(yi=logRR, sei=SElogRR, mods=cbind(e1,e2,e3,e4,e5,e6), 
intercept=FALSE, method="DL", data=a)

Mixed-Effects Model (k = 52; tau^2 estimator: DL)

tau^2 (estimated amount of residual heterogeneity):     0.0805 (SE = 0.0387)
tau (square root of estimated tau^2 value):             0.2837
I^2 (residual heterogeneity / unaccounted variability): 99.46%
H^2 (unaccounted variability / sampling variability):   185.55

Test for Residual Heterogeneity: 
QE(df = 46) = 8535.3517, p-val < .0001

Test of Moderators (coefficient(s) 1,2,3,4,5,6): 
QM(df = 6) = 314.3132, p-val < .0001

Model Results:

    estimate      se      zval    pval    ci.lb    ci.ub     
e1   -0.1973  0.0775   -2.5476  0.0108  -0.3491  -0.0455    *
e2   -0.1679  0.0825   -2.0358  0.0418  -0.3296  -0.0063    *
e3   -0.0413  0.0774   -0.5327  0.5942  -0.1930   0.1105     
e4   -0.1241  0.0990   -1.2539  0.2099  -0.3181   0.0699     
e5   -0.4916  0.1320   -3.7239  0.0002  -0.7503  -0.2329  ***
e6   -1.6907  0.1064  -15.8893  <.0001  -1.8992  -1.4821  ***

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

> rma.uni(yi=logRR, sei=SElogRR, mods=cbind(e1,e2,e3,e4,e5,e6), 
intercept=FALSE, method="REML", data=a)  

Mixed-Effects Model (k = 52; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):     0.1901 (SE = 0.0411)
tau (square root of estimated tau^2 value):             0.4360
I^2 (residual heterogeneity / unaccounted variability): 99.77%
H^2 (unaccounted variability / sampling variability):   437.00

Test for Residual Heterogeneity: 
QE(df = 46) = 8535.3517, p-val < .0001

Test of Moderators (coefficient(s) 1,2,3,4,5,6): 
QM(df = 6) = 142.3333, p-val < .0001

Model Results:

    estimate      se      zval    pval    ci.lb    ci.ub     
e1   -0.1987  0.1175   -1.6913  0.0908  -0.4289   0.0316    .
e2   -0.1695  0.1243   -1.3638  0.1726  -0.4132   0.0741     
e3   -0.0491  0.1165   -0.4212  0.6736  -0.2774   0.1793     
e4   -0.1184  0.1502   -0.7880  0.4307  -0.4127   0.1760     
e5   -0.5140  0.1954   -2.6304  0.0085  -0.8970  -0.1310   **
e6   -1.6950  0.1588  -10.6722  <.0001  -2.0063  -1.3837  ***

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 


For completeness, I checked the other heterogeneity estimators available in 
metafor. Most are close to REML, expect Hunter-Schmidt (HS) which gives an 
even lower tau^2 than DL.

     method  tau2    I2     H2
DL       DL 0.080 99.46 185.55
HS       HS 0.036 98.81  84.07
HE       HE 0.190 99.77 435.58
ML       ML 0.167 99.74 384.58
REML   REML 0.190 99.77 437.00
EB       EB 0.190 99.77 437.01
PM       PM 0.190 99.77 437.01
SJ       SJ 0.195 99.78 447.29

Thank you very much,

Theodore Lytras