[R-meta] Question on Multivariate-Multilevel Meta-Analyis

[Martin Brunner]

Dear meta-analysis folks,



I would like to meta-analyze the results from a regression model where I explain reading achievement by gender, SES, and the interaction between SES and gender. My data stem from seven cycles of a large-scale assessment program with data from 96 countries, up to seven assessment cycles per country and about 4,500 students per cycle within each country. Of note, each student participated only in one single cycle.



My key effect sizes are three (unstandardized) regression coefficients (b_gender, b_SES, b_interaction). I also computed the variance-covariance matrix of the sampling variances/covariances for these three effect sizes for each cycle within each country. I have a total of 412 effect sizes for each regression coefficient which I would like to meta-analyze.



One possibility to meta-analyze these data is to meta-analyze each regression coefficient separately, for example using the rma.mv function of metafor with b_gender depicting the regression coefficient for gender and v_b_gender depicting the sampling variance of this regression coefficient. The metafor code looked like this:



res_b_gender <- rma.mv(yi = b_gender, V = v_b_gender, random = ~ 1 | country/cycle, data = dat, method="REML")



This code works very pretty well.



However, following the literature to summarize regression coefficients (e.g., Becker & Wu, 2007; Gasparrinni et al., 2012) I wondered whether I can also jointly meta-analyze the three regression coefficients by using a multivariate multilevel meta-analytic model. Notably, I would also like to allow each regression coefficient to vary across cycles within countries as well as across countries. Using the multivariate model may further improve the precision of estimates when I correctly understood the literature cited above. For the multivariate model, I computed the block diagonal variance-covariance matrix of the sampling variance-covariance matrix for each combination of a certain cycle within a certain country (depicted by variable cnt_cycle) using a data set in a long-format. The variances and covariances were stored in the variables v1i, v2i, and v3i. The estimates of the regression coefficients were stored in the variable yi and the type of independent variable (i.e, gender, SES, interaction) was depicted in the variable b.



V <- bldiag(lapply(split(dat[,c("v1i", "v2i", "v3i")], dat$cnt_cycle), as.matrix))



Using this matrix I tried to apply the rma.va function with the following code:



res <- rma.mv(yi = yi, V = V, mods = ~ b - 1, random = ~ b | country/cycle, struct="UN", data=dat, method="REML")



Unfortunately, the code does not work. I get the following error message: "Error in rma.mv(yi, V, mods = ~b - 1, random = ~b | country/cycle,  :  Cannot use '~ inner | outer1/outer2' type terms in the 'random' argument."



I guess that I am missing something really important here. Any help or suggestions are highly appreciated.



Best wishes,



Martin







Here is also some code to reproduce my problem taken from the metafor site (http://www.metafor-project.org/doku.php/analyses:berkey1998). I duplicated the data to simulate two assessment cycles (e.g., an initial result [cycle = 1] and a replication attempt [cycle = 2]) within each trial (which corresponds to countries in my data).



library(metafor)

dat1 <- dat.berkey1998

dat1$cycle <- 1



dat2 <- dat.berkey1998

dat2$cycle <- 2



#duplicate data

dat <- rbind(dat1, dat2)



dat$trial_cycle <- paste(dat$trial,dat$cycle)





V <- bldiag(lapply(split(dat[,c("v1i", "v2i")], dat$trial_cycle), as.matrix))



res <- rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial/cycle, struct="UN", data=dat, method="ML")

print(res, digits=3)







References:



Becker, B. J., & Wu, M.-J. (2007). The Synthesis of Regression Slopes in Meta-Analysis. Statistical Science, 22(3), 414-429. https://doi.org/10.1214/07-STS243

Gasparrini, A., Armstrong, B., & Kenward, M. G. (2012). Multivariate meta-analysis for non-linear and other multi-parameter associations. Statistics in Medicine, 31(29), 3821-3839. https://doi.org/10.1002/sim.5471





res <- rma.mv(yi, V, mods = ~ b - 1, random = ~ b | as.factor(cnt2), struct="UN", data=dat, method="REML")

print(res1, digits=3)




	[[alternative HTML version deleted]]