# [R-meta] residual heterogeneity in meta-regression

Viechtbauer, Wolfgang (SP) wo||g@ng@v|echtb@uer @end|ng |rom m@@@tr|chtun|ver@|ty@n|
Tue Mar 24 10:41:23 CET 2020

```Dear Daniel,

In my opinion, the best way to express how much heterogeneity there is is to compute a prediction interval for the true effects/outcomes. In its most basic form, this is computed for a random-effects model with:

mu_hat +- 1.96 * sqrt(tau^2_hat),

where mu_hat and tau^2_hat are just the estimates of mu (the average true effect/outcome) and tau^2 (the variance in the true effects/outcomes) and 1.96 is the 97.5% percentile of a standard normal distribution (rounded).

A slightly improved version takes the uncertainty of mu_hat into consideration:

mu_hat +- 1.96 * sqrt(SE[mu_hat]^2 + tau^2_hat),

where SE[mu] is the standard error of mu_hat. This is in fact how predict() computes this interval. See:

http://www.metafor-project.org/doku.php/faq#for_random-effects_models_fitt

If the 'Knapp and Hartung method' was used when fitting the model, instead of 1.96, the corresponding percentile of a t-distribution with df=k-1 is used.

This interval can be interpreted in various ways. It can be thought of as an interval that is meant to capture 95% of the true effects/outcomes (in the population of studies). It can also be thought of as an interval that is meant to capture the true effect/outcome of a new study (from the same population of studies) with 95% probability. This is very useful information, because it directly tells us how much true effects/outcomes can vary (in the actual units of the outcome/effect size measure). For example, it might be the case that mu_hat is positive and significantly different from 0 (i.e., true effects/outcomes are *on average* positive), but the prediction interval is so wide such that negative true effects/outcomes *in particular studies* are quite plausible.

I emphasize 'meant to' because, strictly speaking, the interval (computed in the forms above and also when using df=k-2) is not wide enough to really have a 95% capture probability. But for its intended purpose -- to provide a range that reflects how large/small the true effects/outcomes in particular studies might be -- it should be good enough.

For meta-regression models, one can compute such prediction intervals conditional on a particular combination of moderator values (e.g., for studies where mod1 = <something>, mod2 = <something>, ...).

Best,
Wolfgang

-----Original Message-----
From: Daniel Mønsted Shabanzadeh [mailto:dmshaban using gmail.com]
Sent: Tuesday, 24 March, 2020 10:11
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: residual heterogeneity in meta-regression

Dear Wolfgang

Picking up the thread once more...Regarding the many studies included in the meta-regression model (up to 450 studies) I do expect very large Q statistics and cannot aim to explain all heterogeneity through meta-regression. Is there a better way to express heterogeneity in this case? I have read that the tau is a better way, however I am not sure of the interpretation and what I may conclude regarding the model and heterogeneity using tau.

Regards,
Daniel

On Mon, Nov 18, 2019 at 4:27 PM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
Dear Daniel,

Roughly, the QE-test statistic should tend to decrease when R^2 is large. Whether this is strictly true depends on how tau^2 is being estimated. However, the QE-test could very well be significant even if R^2 is large. It simply means that there is still a significant amount of residual heterogeneity left.

Best,
Wolfgang

-----Original Message-----
From: Daniel Mønsted Shabanzadeh [mailto:dmshaban using gmail.com]
Sent: Thursday, 14 November, 2019 11:09
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
Subject: residual heterogeneity in meta-regression

Dear Wolfgang

I am performing a meta-regression on multiple one-arm non-randomised studies in order to explore the impact of moderators on complications following a surgical intervention. Adding moderators (age catrgory of the patient, surgical technique etc.) increases the R2:

Mixed-Effects Model (k = 183; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):     0.0074 (SE = 0.0010)
tau (square root of estimated tau^2 value):             0.0860
I^2 (residual heterogeneity / unaccounted variability): 99.20%
H^2 (unaccounted variability / sampling variability):   125.63
R^2 (amount of heterogeneity accounted for):            36.78%

Test for Residual Heterogeneity:
QE(df = 156) = 4730.2255, p-val < .0001

Test of Moderators (coefficient(s) 2:27):
QM(df = 26) = 115.5671, p-val < .0001

However, I am unaware of how to interpretate the rising R2 when QE tests keep on beeing significant. So far I understand that the rising R2 indicates that the heterogeneity is beeing partly explained by moderators, however does QE change in case most heterogeneity is explained by moderators?

Regards,
Daniel