[R-meta] Recommendations on quantifying heterogeneity in three-level meta-regression
jens.schueler at wiwi.uni-kl.de
Tue Nov 21 10:01:12 CET 2017
thank you very much for the honest clarification of the current situation on
Spending a weekend pondering over these issues, I was really left unsure
about what is appropriate/applicable.
Now I have to see what reviewers expect or how they feel about this.
Von: Viechtbauer Wolfgang (SP)
[mailto:wolfgang.viechtbauer at maastrichtuniversity.nl]
Gesendet: Donnerstag, 16. November 2017 21:38
An: Jens Schüler <jens.schueler at wiwi.uni-kl.de>;
r-sig-meta-analysis at r-project.org
Betreff: RE: Recommendations on quantifying heterogeneity in three-level
Maybe Mike can also chime in here, but I don't think metaSEM and metafor do
anything fundamentally different here. Consider this example:
dat <- get(data(dat.konstantopoulos2011))
dat$year <- dat$year - 1976
res0 <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat,
res1 <- rma.mv(yi, vi, mods = ~ year, random = ~ 1 | district/school,
### pseudo R^2 values
round(max(0, (res0$sigma2 - res1$sigma2) / res0$sigma2), 4) ###
level 3 round(max(0, (res0$sigma2 - res1$sigma2) / res0$sigma2), 4)
### level 2
res2 <- meta3(yi, vi, x=year, cluster=district, data=dat)
It takes a bit of time to match things up, but you will find that the
results are pretty much the same.
I agree with Mike that one needs to be cautious about computing 'I^2-like'
measures for such models. The code on the metafor website (i.e.,
ilevel_models) is pretty much a logical extension of one way of how I^2 can
be computed for standard RE models and it is there because I have received
many questions on how to compute I^2 for more complex models, but there is
no consensus yet as to how this should be done. Heck, there isn't even
consensus on how to compute I^2 for standard RE models!
Also, the interpretation of I^2-like measures computed based on models that
include moderators/predictors is somewhat nebulous (to the point that one
could question whether it is even useful to compute such things).
Wolfgang Viechtbauer, Ph.D., Statistician | Department of Psychiatry and
Neuropsychology | Maastricht University | P.O. Box 616 (VIJV1) | 6200 MD
Maastricht, The Netherlands | +31 (43) 388-4170 | http://www.wvbauer.com
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>project.org] On Behalf Of Jens Schüler
>Sent: Thursday, 16 November, 2017 14:26
>To: r-sig-meta-analysis at r-project.org
>Subject: [R-meta] Recommendations on quantifying heterogeneity in
>three- level meta-regression
>I am currently working on a three-level meta-regression and I am a bit
>unsure about how to properly/adequately report heterogeneity in such
>mixed-effect models (whereas it is pretty clear cut for regular "two-
>level" meta-regressions e.g. the recent best practice recommendations
>of Gonzalez-Mule & Aguinis 2017
>Currently the metafor rma.mv function provides us with variance
>estimates for both levels and a Q-test on residual heterogeneity.
>Moreover, we are provided with code on how to calculate I2 in
>multilevel models, which I did (http://www.metafor-
>Turning towards the metaSEM package of Cheung, he reports variance
>estimates (no predictors vs. predictors) for both levels and R2
>#m odel-1-type-as-a-covariate). However, not I2. In his book (Cheung
>2015) reports, on page 186, some formulas on how to calculate I2 and
>ICC in three-level models but ends with a cautionary note that we lack
>further insight in order to make reliable statements on that matter.
>Hence, I am a bit left in the dark in terms of what is currently
>appropriate (or how to defend such choices) and would like to pick your
>brains on that issue.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 7675 bytes
Desc: not available
More information about the R-sig-meta-analysis