[R-meta] prediction intervals in multi-level meta-analysis models

[Viechtbauer, Wolfgang (SP)]

Hi James,

Indeed, that's what predict() does for such a model (and this extends to models with more levels). There is, as far as I know, no reference for this. The method is just the logical extension of the way prediction intervals are computed for 'standard' RE/ME models (leaving aside the issue of whether to use a critical z/t value and the dfs for the t value when using the latter).

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of James Pustejovsky
>Sent: Wednesday, 14 April, 2021 17:40
>To: R meta
>Subject: [R-meta] prediction intervals in multi-level meta-analysis models
>
>Dear meta-analysis folks,
>
>I am using metafor to fit a multi-level meta-regression model. I want to
>generate an 80% prediction interval for the effect size in a new study with
>a given set of characteristics. Here's an example using the Konstantopoulos
>data on effects modified school calendars:
>
>library(metafor)
>data("dat.konstantopoulos2011")
>
>kon_fit <- rma.mv(yi = yi, V = vi, data = dat.konstantopoulos2011,
>                  mods = ~ year,
>                  random = ~ 1 | district / school,
>                  sparse = TRUE)
>
>predict(kon_fit, newmods = 2021, level = 80)
>
>(This particular example is a bit extreme because it involves a massive
>extrapolation forward in time, but this isn't relevant to my question.)
>
>It looks like the prediction interval generated by metafor::predict()
>incorporates three sources of variation: 1) uncertainty in the average
>effect size estimate at the given value of the predictor, 2) estimated
>between-district variation, and 3) estimated school-level variation.
>
>My question is: does anyone know of any references on the methodology
>behind constructing such prediction intervals (involving multiple variance
>components)? The methods implemented in metafor seem very sensible, but I
>can't seem to find anything in the literature about this stuff.
>
>Kind Regards,
>James