[R-meta] R2 in rma.mv() with missing pre-tests

[Viechtbauer, Wolfgang (NP)]

Dear Zhouhan,

Please post in plain text. Note below how HTML emails end up messing with the text (e.g., turning rma.mv into a URL for a non-existing website in the Maldives).

Just for reference, this post seems to be related to this thread:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2024-March/005095.html

So, to summarize: You ended up computing SMDs quantifying the difference between two groups at two time points (with some studies allowing for the computation of SMD values for both time points), then meta-analyzed those values via a (multilevel) meta-regression with time (and some other moderator) as predictor, and based on this model compute contrasts indicating how the pooled SMD value at the two time points differ (as a reflection of the 'gain'). I would still be a bit cautious with this approach, since you are not directly computing the (standardized) change within groups, but let's leave this aside.

I am not sure how you are computing R^2 here in the first place (maybe based on the proportional reduction in the variance components when including 'mod' in the model). If so, I would say that this type of pseudo-R^2 values does tell you something about how much heterogeneity is accounted for by the moderator. But is this 'heterogeneity in the gains'? Not quite I would say, since the variance components of the model do not reflect such heterogeneity (since you are meta-analyzing SMD values at particular time points).

Maybe you could contrast this approach with the one where you just use the ~60% studies that do have a pre-test (and where you can directly compute SMC values).

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Zhouhan Jin via R-sig-meta-analysis
> Sent: Saturday, May 4, 2024 21:48
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Cc: Zhouhan Jin <zjin65 using uwo.ca>
> Subject: [R-meta] R2 in rma.mv() with missing pre-tests
>
> Hello all,
>
> I have ~40% of my longitudinal studies missing pre-tests. As a result, I can't
> compute the gain effects (ex. SMCCs) directly from these studies or if I do, I
> will lose them.
>
> Out of necessity, I meta-analyzed the effects at each time (i.e., SMDs) using
> the best empirically fitting model:
>
> MODEL <- rma.mv<http://rma.mv/>(yi ~ time * mod, V = V, random =
> ~1|study/effect)
>
> and then computed the gains posthoc using the emmeans package:
>
> gain <- contrast(emmprep(MODEL), list(c(1,-1,0,0))) and similar for various
> categories of `mod`.
>
> Question: Am I right that R2 from this MODEL can't tell us the extent `mod`
> explains the heterogeneity in gains?
>
> If not, what alternatives do I have to get an insight into the heterogeneity in
> the gains explained by `mod`?
>
> Thanks in advance!
>
> Best wishes,
>
> Zhouhan