[R-meta] R2 in rma.mv() with missing pre-tests

[Zhouhan Jin]

Thank you, Wolfgang! You mentioned "I am not sure how you are computing R^2 here in the first place (maybe based on the proportional reduction in the variance components when including 'mod' in the model)." Do you think the null model used to calculate R^2 should be: "yi ~ 1" or "yi ~ time" when examining the role of `mod`?

You also mentioned "Not quite I would say" regarding if R^2 from MODEL showing 'heterogeneity in the gains'. Do you mean R^2 kind of is that, if we think of gain as SMDRP instead of SMCC?

Thanks again!

Best wishes,

Zhouhan

On May 6, 2024 at 07:10 -0400, Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>, wrote:
Dear Zhouhan,

Please post in plain text. Note below how HTML emails end up messing with the text (e.g., turning rma.mv into a URL for a non-existing website in the Maldives).

Just for reference, this post seems to be related to this thread:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2024-March/005095.html

So, to summarize: You ended up computing SMDs quantifying the difference between two groups at two time points (with some studies allowing for the computation of SMD values for both time points), then meta-analyzed those values via a (multilevel) meta-regression with time (and some other moderator) as predictor, and based on this model compute contrasts indicating how the pooled SMD value at the two time points differ (as a reflection of the 'gain'). I would still be a bit cautious with this approach, since you are not directly computing the (standardized) change within groups, but let's leave this aside.

I am not sure how you are computing R^2 here in the first place (maybe based on the proportional reduction in the variance components when including 'mod' in the model). If so, I would say that this type of pseudo-R^2 values does tell you something about how much heterogeneity is accounted for by the moderator. But is this 'heterogeneity in the gains'? Not quite I would say, since the variance components of the model do not reflect such heterogeneity (since you are meta-analyzing SMD values at particular time points).

Maybe you could contrast this approach with the one where you just use the ~60% studies that do have a pre-test (and where you can directly compute SMC values).

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
Of Zhouhan Jin via R-sig-meta-analysis
Sent: Saturday, May 4, 2024 21:48
To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
project.org>
Cc: Zhouhan Jin <zjin65 using uwo.ca>
Subject: [R-meta] R2 in rma.mv() with missing pre-tests

Hello all,

I have ~40% of my longitudinal studies missing pre-tests. As a result, I can't
compute the gain effects (ex. SMCCs) directly from these studies or if I do, I
will lose them.

Out of necessity, I meta-analyzed the effects at each time (i.e., SMDs) using
the best empirically fitting model:

MODEL <- rma.mv<http://rma.mv/>(yi ~ time * mod, V = V, random =
~1|study/effect)

and then computed the gains posthoc using the emmeans package:

gain <- contrast(emmprep(MODEL), list(c(1,-1,0,0))) and similar for various
categories of `mod`.

Question: Am I right that R2 from this MODEL can't tell us the extent `mod`
explains the heterogeneity in gains?

If not, what alternatives do I have to get an insight into the heterogeneity in
the gains explained by `mod`?

Thanks in advance!

Best wishes,

Zhouhan

	[[alternative HTML version deleted]]