[R-meta] Question on Multivariate-Multilevel Meta-Analyis

[Viechtbauer, Wolfgang (SP)]

Dear Martin,

It is a bit of an annoying limitation that rma.mv() does not allow terms like

random = ~ inner | outer1/outer2

but one can easily work around that. Just use:

res <- rma.mv(yi = yi, V = V, mods = ~ b - 1, random = list(~ b | country, ~ b | interaction(country,cycle)), struct=c("UN","UN"), data=dat, method="REML")

Maybe I will clean up the code in rma.mv() eventually so that 'random = ~ inner | outer1/outer2' is possible, but it's not urgent as far as I am concerned given this easy workaround.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Martin Brunner
>Sent: Thursday, 29 April, 2021 18:22
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Question on Multivariate-Multilevel Meta-Analyis
>
>Dear meta-analysis folks,
>
>I would like to meta-analyze the results from a regression model where I explain
>reading achievement by gender, SES, and the interaction between SES and gender. My
>data stem from seven cycles of a large-scale assessment program with data from 96
>countries, up to seven assessment cycles per country and about 4,500 students per
>cycle within each country. Of note, each student participated only in one single
>cycle.
>
>My key effect sizes are three (unstandardized) regression coefficients (b_gender,
>b_SES, b_interaction). I also computed the variance-covariance matrix of the
>sampling variances/covariances for these three effect sizes for each cycle within
>each country. I have a total of 412 effect sizes for each regression coefficient
>which I would like to meta-analyze.
>
>One possibility to meta-analyze these data is to meta-analyze each regression
>coefficient separately, for example using the rma.mv function of metafor with
>b_gender depicting the regression coefficient for gender and v_b_gender depicting
>the sampling variance of this regression coefficient. The metafor code looked like
>this:
>
>res_b_gender <- rma.mv(yi = b_gender, V = v_b_gender, random = ~ 1 |
>country/cycle, data = dat, method="REML")
>
>This code works very pretty well.
>
>However, following the literature to summarize regression coefficients (e.g.,
>Becker & Wu, 2007; Gasparrinni et al., 2012) I wondered whether I can also jointly
>meta-analyze the three regression coefficients by using a multivariate multilevel
>meta-analytic model. Notably, I would also like to allow each regression
>coefficient to vary across cycles within countries as well as across countries.
>Using the multivariate model may further improve the precision of estimates when I
>correctly understood the literature cited above. For the multivariate model, I
>computed the block diagonal variance-covariance matrix of the sampling variance-
>covariance matrix for each combination of a certain cycle within a certain country
>(depicted by variable cnt_cycle) using a data set in a long-format. The variances
>and covariances were stored in the variables v1i, v2i, and v3i. The estimates of
>the regression coefficients were stored in the variable yi and the type of
>independent variable (i.e, gender, SES, interaction) was depicted in the variable b.
>
>V <- bldiag(lapply(split(dat[,c("v1i", "v2i", "v3i")], dat$cnt_cycle), as.matrix))
>
>Using this matrix I tried to apply the rma.va function with the following code:
>
>res <- rma.mv(yi = yi, V = V, mods = ~ b - 1, random = ~ b | country/cycle,
>struct="UN", data=dat, method="REML")
>
>Unfortunately, the code does not work. I get the following error message: "Error
>in rma.mv(yi, V, mods = ~b - 1, random = ~b | country/cycle,  :  Cannot use '~
>inner | outer1/outer2' type terms in the 'random' argument."
>
>I guess that I am missing something really important here. Any help or suggestions
>are highly appreciated.
>
>Best wishes,
>
>Martin
>
>Here is also some code to reproduce my problem taken from the metafor site
>(http://www.metafor-project.org/doku.php/analyses:berkey1998). I duplicated the
>data to simulate two assessment cycles (e.g., an initial result [cycle = 1] and a
>replication attempt [cycle = 2]) within each trial (which corresponds to countries
>in my data).
>
>library(metafor)
>dat1 <- dat.berkey1998
>dat1$cycle <- 1
>
>dat2 <- dat.berkey1998
>dat2$cycle <- 2
>
>#duplicate data
>dat <- rbind(dat1, dat2)
>dat$trial_cycle <- paste(dat$trial,dat$cycle)
>
>V <- bldiag(lapply(split(dat[,c("v1i", "v2i")], dat$trial_cycle), as.matrix))
>
>res <- rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial/cycle,
>struct="UN", data=dat, method="ML")
>
>print(res, digits=3)
>
>References:
>
>Becker, B. J., & Wu, M.-J. (2007). The Synthesis of Regression Slopes in Meta-
>Analysis. Statistical Science, 22(3), 414-429. https://doi.org/10.1214/07-STS243
>
>Gasparrini, A., Armstrong, B., & Kenward, M. G. (2012). Multivariate meta-analysis
>for non-linear and other multi-parameter associations. Statistics in Medicine,
>31(29), 3821-3839. https://doi.org/10.1002/sim.5471
>
>res <- rma.mv(yi, V, mods = ~ b - 1, random = ~ b | as.factor(cnt2), struct="UN",
>data=dat, method="REML")
>
>print(res1, digits=3)