[R-meta] Test statistics for variance components in multivariate meta-analysis and interpretation of the output

[Viechtbauer, Wolfgang (SP)]

Hi Selcuk,

Please see below for my responses.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Acar, Selcuk
>Sent: Wednesday, 07 July, 2021 18:02
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Test statistics for variance components in multivariate meta-
>analysis and interpretation of the output
>
>Hi,
>
>I am using metafor package for my three level (study, sample, effect size levels)
>multivariate meta-analysis. I was trying to figure out how to get confidence
>intervals and I think I figured out. I could get the output below via "confint".
>However, there are two things I need to know:
>
>1) How do I obtain test statistics for the variance components? Is there a
>practical way to do that in metafor package?

You can do likelihood ratio tests. See help(anova.rma). For example:

dat <- dat.konstantopoulos2011
res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)

### test the district-level variance component
res0 <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat, sigma2=c(0,NA))
anova(res, res0)

### test the school-level variance component
res0 <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat, sigma2=c(NA,0))
anova(res, res0)

### test both variance components simultaneously
res0 <- rma.mv(yi, vi, data=dat)
anova(res, res0)

This can be easily extended to models with more than two variance components.

>2) I know that sigma^2.1, sigma^2.2 and sigma^2.3 are the variance components at
>three different levels. But, I do not know what "sigma.1" "sigma.2" and "sigma.3" 
>represent (see the output below).
>
> confint(MVmodel_null, digits=3)
>
>          estimate ci.lb ci.ub
>sigma^2.1    0.006 0.000 0.020
>sigma.1      0.080 0.000 0.143
>
>          estimate ci.lb ci.ub
>sigma^2.2    0.019 0.008 0.033
>sigma.2      0.137 0.091 0.181
>
>          estimate ci.lb ci.ub
>sigma^2.3    0.011 0.009 0.013
>sigma.3      0.106 0.097 0.115

Those are just the square-root of the variances (i.e., the SDs).

>Can you please help with those questions?
>
>Selcuk Acar, Ph.D.
>Associate Professor
>Department of Educational Psychology
>University of North Texas