[R-meta] Two ways to calculate subgroup and overall average effect sizes

[Viechtbauer Wolfgang (SP)]

You can do a LRT to compare the models. But you have to fit the 'standard' model also with rma.mv(). See here:

http://www.metafor-project.org/doku.php/tips:rma.uni_vs_rma.mv

So, in your case:

painearly.df$id <- 1:nrow(painearly.df)

painearly1surgery.rma <- rma.mv(yi = yi, V = vi, mods = ~ surgery, random = ~ 1 | id,
                            data = painearly.df, test = 't', digits = 3)

painearly2surgery.rma.mv <- rma.mv(yi = yi, V = vi, mods = ~ surgery,
                         random = ~ surgery | study, struct = 'DIAG',
                         data = painearly.df, test = 't', digits = 3)

anova(painearly1surgery.rma, painearly2surgery.rma.mv)

So, that will give you a LRT of H0: tau^2.1 = tau^2.2 = tau^2.3.

Another way to think about this is that this is a comparison between an 'ID' and a 'DIAG' structure.

res0 <- rma.mv(yi = yi, V = vi, mods = ~ surgery,
               random = ~ surgery | study, struct = 'ID',
               data = painearly.df, test = 't', digits = 3)

res1 <- rma.mv(yi = yi, V = vi, mods = ~ surgery,
               random = ~ surgery | study, struct = 'DIAG',
               data = painearly.df, test = 't', digits = 3)

anova(res0, res1)

But that will give you the same results (assuming that there are no studies where the same level of 'surgery' occurs more than once).

Best,
Wolfgang

-----Original Message-----
From: Nathan Pace [mailto:n.l.pace at utah.edu] 
Sent: Thursday, August 24, 2017 00:44
To: Viechtbauer Wolfgang (SP); r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Two ways to calculate subgroup and overall average effect sizes

Hi Wolfgang, 

I have a k = 29 SMD meta analysis.

The moderator is a three level factor.

painearly1surgery.rma <- rma(yi = yi, vi = vi, mods = ~ surgery,
                            data = painearly.df, test = 'knha', digits = 3)

painearly2surgery.rma.mv <- rma.mv(yi = yi, V = vi, mods = ~ surgery,
                         random = ~ surgery | study, struct = 'DIAG',
                         data = painearly.df, test = 't', digits = 3)

There is a nearly 10 fold variation in the individual tau^2s.

Variance Components: 

outer factor: study   (nlvls = 29)
inner factor: surgery (nlvls = 3)

           estim   sqrt  k.lvl  fixed  level
tau^2.1    0.082  0.287      8     no   open
tau^2.2    0.759  0.871     10     no    lap
tau^2.3    0.086  0.294     11     no  other

The average tau^2 is:

tau^2 (estimated amount of residual heterogeneity):     0.312 (SE = 0.110)
tau (square root of estimated tau^2 value):             0.559

The omnibus test of moderators  is not rejected in either model.

Test of Moderators (coefficient(s) 2:3):  (average tau^2)
F(df1 = 2, df2 = 26) = 2.082, p-val = 0.145

Test of Moderators (coefficient(s) 2:3):  (individual tau^2)
F(df1 = 2, df2 = 26) = 2.643, p-val = 0.090

Is there a meaningful statistical comparison of the individual tau^2s?

Are there other ways to compare the model fits (AIC or BIC)?

The anova function won’t mix rma.uni and rma.mv objects.

Nathan