[R-meta] Interpreting 95%CI estimated in the multimodel inference via glmulti

[Viechtbauer, Wolfgang (SP)]

Dear Gabri,

To get (approximate) 95% CIs, compute 'Estimate +- 1.96 * sqrt(Uncond. variance)'. So, using the intercept values, for "pt:h", this would be:

-0.1065 + c(-1,1) * 1.96 * sqrt(0.0021)

For "pt:w", this would be:

-0.0117 + c(-1,1) * 1.96 * sqrt(0.0010)

Note that the intercept is the estimated average true outcome when all predictors are equal to 0. Whether this is a meaningful value depends on how the predictors ('sol', 'ai', etc.) are coded.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Gabriele Midolo
Sent: Wednesday, 05 September, 2018 11:11
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] Interpreting 95%CI estimated in the multimodel inference via glmulti

Dear all,

Following Wolfgang instructions on multimodel inference with glmulti (
http://www.metafor-project.org/doku.php/tips:model_selection_with_glmulti),
I came out with the following output in my analysis:

> output_SLA        Estimate Uncond. variance Nb models Importance +/- (alpha=0.05)
sol       0.0029           0.0005        15     0.2776           0.0421
ai        0.0133           0.0007        16     0.3906           0.0512
ele      -0.0172           0.0006        16     0.4936           0.0472
mat      -0.0383           0.0007        23     0.8078           0.0524
ptw       0.0948           0.0037        23     0.8322           0.1188
ldele    -0.0282           0.0002        27     0.9092           0.0289
intrcpt  -0.1065           0.0021        34     1.0000           0.0888

I am struggling a bit to understand alpha value applied to the categorical
moderator I have in the meta-regression i.e. 'pt'. The 'pt' variable can
assume two values: "pt:w" and "pt:h", the latter is the intrcpt in
'output_SLA'. When I re-run the model selection by making 'ptw' the
intercept I get the value that is consistent with the 'Estimate' of
'output_SLA' (i.e.  pt:w calculated as 0.0948 - 0.1065 = -0.0117). But the
alpha values of the intercpt changed (now = 0.0618):

> output_SLA2                 Estimate Uncond. variance Nb models Importance +/- (alpha=0.05)
sol                0.0029           0.0005        15     0.2776           0.0421
ai                 0.0133           0.0007        16     0.3906           0.0512
ele               -0.0172           0.0006        16     0.4936           0.0472
mat               -0.0383           0.0007        23     0.8078           0.0524
pth               -0.0948           0.0037        23     0.8322           0.1188
ldele             -0.0282           0.0002        27     0.9092           0.0289
intrcpt           -0.0117           0.0010        34     1.0000           0.0618

My question is then how to estimate 95%CI of the mean pooled effect size
for pt:h and pt:w from both models? Should I add/subtract the alpha to the
Estimate of ptw/pth and then add it to the CI of intercept? Or should I
estimate alpha directly by adding pth/ptw to the intercept's alpha? In both
cases I think I end up with different 95%CI estimated for the two
categories depending on which one is "forced" to be the intercept?

Hope I was clear,
Thanks and cheers,
Gabri