[R-meta] R-square (change) as effect size

[Viechtbauer, Wolfgang (NP)]

Dear Paul,

I think the issue of using R^2 as an effect size measure for a meta-analysis has come up before on this mailing list, although I can't find the threads right now. In any case, there are several practical issues:

1) The sampling distribution of (adjusted) R^2 is not normal, so one needs to figure out some appropriate normalizing transformation.

2) One also needs to figure out the sampling variance of the (transformed) (adjusted) R^2 value.

3) One can also debate the usefulness of meta-analyzing a 'directionless' measure such as (adjusted) R^2. If you focus on the difference in (adjusted) R^2 though (of two non-nested models), then I think this issue is less concerning (and the sampling distribution of such a difference might actually be somewhat normal). However, this then raises another problem: The two (transformed) (adjusted) R^2 values are not independent if they come from the same sample and so one now also needs to figure out their covariance. If they do not come from the same sample, then this alleviates this particular issue, but makes the evidence much weaker due to potential confounding.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Hanel, Paul H P via R-sig-meta-analysis
>Sent: Thursday, 13 April, 2023 16:53
>To: r-sig-meta-analysis using r-project.org
>Cc: Hanel, Paul H P
>Subject: [R-meta] R-square (change) as effect size
>
>Hello,
>
>Is it possible to run a meta-analysis using R-square (or R) as an effect size in
>the same way as you run a meta-analysis using Pearson's r?
>
>Specifically, I am interested in which of two sets of psychological constructs is
>better in predicting a range of outcomes such as well-being or self-reported
>behaviour. Set 1 consists of five variables (the so-called Big-5 personality
>traits) whereas set 2 consists of 10 variables (Schwartz's 10 value types). I
>have the adjusted R-squares from linear regressions for both sets of variables as
>well as the adjusted R-square change. For example, does set 1 explain more
>variance in well-being than set 2?
>
>Thanks
>Paul