[R-meta] Interpreting 95%CI estimated in the multimodel inference via glmulti

[Viechtbauer, Wolfgang (SP)]

Ah, I just realized that this is also what I wrote up here:

http://www.metafor-project.org/doku.php/tips:model_selection_with_glmulti#multimodel_inference

"The model-averaged parameter estimates and the unconditional variances can be used for multimodel inference. For example, adding and subtracting the values in the last column from the model-averaged parameter estimates yields approximate 95% confidence intervals for each coefficient that are based not on any one model, but all models in the candidate set."

So yes, you can just take the values from the "(alpha=0.05)" column.

Best,
Wolfgang

-----Original Message-----
From: Gabriele Midolo [mailto:gabriele.midolo using gmail.com] 
Sent: Monday, 10 September, 2018 17:05
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] Interpreting 95%CI estimated in the multimodel inference via glmulti

Ok thanks Wolfgang,
I actually noted it is the same of simply adding/subtratting the 'alpha=0.05' term to the estimate (?).
Other predictors are scaled in the regression and thereby the intercept is the true outcome when all predictors equal to their mean.

Cheers,
Gabri

On 10 September 2018 at 10:46, Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
Dear Gabri,

To get (approximate) 95% CIs, compute 'Estimate +- 1.96 * sqrt(Uncond. variance)'. So, using the intercept values, for "pt:h", this would be:

-0.1065 + c(-1,1) * 1.96 * sqrt(0.0021)

For "pt:w", this would be:

-0.0117 + c(-1,1) * 1.96 * sqrt(0.0010)

Note that the intercept is the estimated average true outcome when all predictors are equal to 0. Whether this is a meaningful value depends on how the predictors ('sol', 'ai', etc.) are coded.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Gabriele Midolo
Sent: Wednesday, 05 September, 2018 11:11
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] Interpreting 95%CI estimated in the multimodel inference via glmulti

Dear all,

Following Wolfgang instructions on multimodel inference with glmulti (
http://www.metafor-project.org/doku.php/tips:model_selection_with_glmulti),
I came out with the following output in my analysis:

> output_SLA        Estimate Uncond. variance Nb models Importance +/- (alpha=0.05)
sol       0.0029           0.0005        15     0.2776           0.0421
ai        0.0133           0.0007        16     0.3906           0.0512
ele      -0.0172           0.0006        16     0.4936           0.0472
mat      -0.0383           0.0007        23     0.8078           0.0524
ptw       0.0948           0.0037        23     0.8322           0.1188
ldele    -0.0282           0.0002        27     0.9092           0.0289
intrcpt  -0.1065           0.0021        34     1.0000           0.0888

I am struggling a bit to understand alpha value applied to the categorical
moderator I have in the meta-regression i.e. 'pt'. The 'pt' variable can
assume two values: "pt:w" and "pt:h", the latter is the intrcpt in
'output_SLA'. When I re-run the model selection by making 'ptw' the
intercept I get the value that is consistent with the 'Estimate' of
'output_SLA' (i.e.  pt:w calculated as 0.0948 - 0.1065 = -0.0117). But the
alpha values of the intercpt changed (now = 0.0618):

> output_SLA2                 Estimate Uncond. variance Nb models Importance +/- (alpha=0.05)
sol                0.0029           0.0005        15     0.2776           0.0421
ai                 0.0133           0.0007        16     0.3906           0.0512
ele               -0.0172           0.0006        16     0.4936           0.0472
mat               -0.0383           0.0007        23     0.8078           0.0524
pth               -0.0948           0.0037        23     0.8322           0.1188
ldele             -0.0282           0.0002        27     0.9092           0.0289
intrcpt           -0.0117           0.0010        34     1.0000           0.0618

My question is then how to estimate 95%CI of the mean pooled effect size
for pt:h and pt:w from both models? Should I add/subtract the alpha to the
Estimate of ptw/pth and then add it to the CI of intercept? Or should I
estimate alpha directly by adding pth/ptw to the intercept's alpha? In both
cases I think I end up with different 95%CI estimated for the two
categories depending on which one is "forced" to be the intercept?

Hope I was clear,
Thanks and cheers,
Gabri