[R-meta] Random Slope Three-Level Meta-Analysis Model in combination with RVE

[Prof. Dr. Martin Brunner]

Dear Listmembers,



I would like to  meta-analyze gender differences in reading achievement. My data stem from seven cycles of a triennial large-scale assessment program with data from 96 countries, up to seven assessment cycles per country and about 4,500 students per cycle within each country. Of note, each student participated only in one single cycle. Nevertheless, the data are in some sense longitudinal because they allow to examine how the size of gender differences developed over time (i.e., cycles) within a certain country.



In my meta-analytic model I would like to examine the linear rate of change in standardized mean differences between female and male students' reading achievement across time. Given that cycles are separated by a three-year time interval, I coded time with 0 for the first cycle, 3 for the second cycle, 6 for the third cycle etc.



Further, some countries implemented educational policies that aimed at reducing gender differences in reading achievement. To simplify things, I assume that the policies were in effect from the very first cycle and did not change over time (i.e., I treat policies as a time-invariant covariate). To this end, I coded policy with (0 = no policy to reduce gender disparities in a certain country, 1 = policy to reduce gender disparities in a certain country). In doing so, I would like to investigate the extent to which the educational policies were related to the linear rate of change of gender differences.



To specify my model I would like to use the rma.mv function of metafor with SMD_gender depicting the standardized mean differences in reading achievement and v_SMD depicting the sampling variance of this effect size. The R code was as follows:



First, I created a variable to represent each available combination of the country variable (cnt) and the time variable.



dat$time_x_cnt <- paste(dat$cnt,dat$time)



Second, the interaction between time and policy represents a cross-level interaction because policy only varies between but not within countries. Following Snijders and Bosker (2012)  I therefore specified a random intercept and a random slope for the time variable that allows both coefficients to vary between countries. Further, I specified a random coefficient for the residual term of the true effect size within countries.



res <- rma.mv(yi = SMD_gender, V = v_SMD,  mods = ~ time*policy,

                               random = list(~ 1 + time | cnt , ~  1 | time_x_cnt),

                              struct=c("GEN","GEN"),

                              data=dat, method="REML" )



In essence I tried to specify a "typical" random-slope growth model to study the cross-level interaction. The code works but I am unsure whether I correctly specified the model.



Further, I would like to take advantage of the RVE method to adjust the standard errors/95% CIs of the meta-regression coefficients. To this end, I used the clubSandwich package.



conf_int(res, vcov = "CR2")



The code works. But I am not sure whether RVE provides robust standard errors for random-slope models.



I highly appreciate any advice and comments.



Best wishes,



Martin





Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel Analysis. An introduction to basic and advanced multilevel modeling (2nd edition). SAGE Publications.




	[[alternative HTML version deleted]]