[R-meta] Prediction intervals for multilevel meta-analysis

[Hanel, Paul H P]

Why do prediction intervals get so much wider when a multi-level approach is used?

Prediction intervals are usually computed by +/- tau*1.96. Obtaining tau is straightforward when doing a random-effects meta-analysis (e.g., function rma() with metafor).

When running a multilevel meta-analysis, things are a bit more complicated. According to Wolfgang Viechtbauer, it is possible to take the sum of the taus � (or sigmas, as the taus are called in the output of the rma.mv() function). However, this results in even wider prediction intervals. For a random effects meta-analysis with over 300 effect sizes, the width of the prediction interval is 1.60 (tau = 0.41). Command used: rma(yi, vi, data = df)
When I run a multilevel meta-analysis with effect sizes nested in studies, the width of the prediction interval is 2.34 (tau/sigma level 1 = .279, level 2 = .317). Command used: rma.mv(yi, vi, random = list(~ 1 | effectID, ~ 1 | StudyID), tdist = TRUE, data=df)
If I add yet another level, articles (i.e., effect sizes nested within studies, studies nested within papers), the width of the prediction interval gets even wider: 2.74 (tau/sigma level 1 = .275, level 2 = .114, level 3 = .309). Command used: rma.mv(yi, vi, random = list(~ 1 | effectID, ~ 1 | StudyID, ~ 1 | PaperID), tdist = TRUE, data=df)

Is it plausible that the prediction intervals get that much wider?

Thanks,
Paul



	[[alternative HTML version deleted]]