[R-meta] Question regarding Generalized Linear Mixed-effects Model for Meta-analysis

[Akifumi Yanagisawa]

Dear all,

I am having some difficulty dealing with proportional data; the dependent variable is learning gain from an activity, in which means and SDs are converted into proportion. The learning gains are nested in each article; each article examined the learning gains from different types of activities and measured the learning gain at different timing (i.e., immediate post and delayed post). The main thing I would like to do is to get the estimated learning gain percentage and its confidence interval for each activity. 

Using the rma.mv() function, I noticed that estimation values go over 100% sometimes; then I thought I should use generalized linear mixed effects model. On the metafor’s webpage (http://www.metafor-project.org/doku.php/todo), I found that the rma.glmm() command does not support Multilevel Models so far and suggested using the LME4 package. I have been trying to figure out how to do this by myself, but I am not sure if I am doing this right. I would appreciate it if you could see if my approach is appropriate and answer to some of my questions.

(1) The approach I tried was, (1) calculated variance from means, SDs, and the numbers of participants by using the escalc function, and (2) then I tried ‘results <- glmer (learning_gain ~ ACTIVITY * TEST_TYPE * TEST_TIMING + (1|article_number/participant_group) + (1|TEST_TIMING:participant_group), weights = 1/vi, family = binomial (link = logit))’. I use the sjPlot package for plotting and the emmeans package to get estiamted learning gain percentages.  Does this sound like the proper approach? Are there other options should I add?

(2) Is it possible for me to get I^2 and H^2 values? I would like to know the proportion of variance explained by each the moderator.

(3) Is there anyway I can conduct (a) Test for Residual Heterogeneity and (b) Test of Moderators? If so, which R package would you recommend? I noticed that the anova function does not provide p-values for the test, and the LmerTest package does not work with the glmer function, either.

Any suggestions and comments will be greatly appreciated. Thank you for your help.

Aki