[R-meta] R-square (change) as effect size

[Viechtbauer, Wolfgang (NP)]

Great, thanks. Also found this thread:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-March/002708.html

which goes a bit in the same direction.

If one focuses on the difference between the two R^2 values, then this might hold some promise, but there are still these pesky little technical details to figure out -- (approximate) normality of the sampling distribution and the sampling variance of such a difference.

P.S.: Just as a reminder to all, there is a custom Google search set up for searching the mailing list archives here: https://cse.google.com/cse?cx=ee4b2e6c93b6a9667 One caveat: Google crawls the archives only periodically, so recent posts will not show up in the search (for example, I tried a search for "R2" and it doesn't bring up the thread from January, but it did lead me to the one from 2021).

Best,
Wolfgang

>-----Original Message-----
>From: Michael Dewey [mailto:lists using dewey.myzen.co.uk]
>Sent: Monday, 17 April, 2023 17:04
>To: R Special Interest Group for Meta-Analysis
>Cc: Viechtbauer, Wolfgang (NP)
>Subject: Re: [R-meta] R-square (change) as effect size
>
>Dear Wolfgang, you were looking for
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-January/004331.html
>and the surrounding thread although my hint there only addressed you
>point 3.
>
>Michael
>
>On 17/04/2023 13:23, Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis
>wrote:
>> Dear Paul,
>>
>> I think the issue of using R^2 as an effect size measure for a meta-analysis
>has come up before on this mailing list, although I can't find the threads right
>now. In any case, there are several practical issues:
>>
>> 1) The sampling distribution of (adjusted) R^2 is not normal, so one needs to
>figure out some appropriate normalizing transformation.
>>
>> 2) One also needs to figure out the sampling variance of the (transformed)
>(adjusted) R^2 value.
>>
>> 3) One can also debate the usefulness of meta-analyzing a 'directionless'
>measure such as (adjusted) R^2. If you focus on the difference in (adjusted) R^2
>though (of two non-nested models), then I think this issue is less concerning
>(and the sampling distribution of such a difference might actually be somewhat
>normal). However, this then raises another problem: The two (transformed)
>(adjusted) R^2 values are not independent if they come from the same sample and
>so one now also needs to figure out their covariance. If they do not come from
>the same sample, then this alleviates this particular issue, but makes the
>evidence much weaker due to potential confounding.
>>
>> Best,
>> Wolfgang
>>
>>> -----Original Message-----
>>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
>On
>>> Behalf Of Hanel, Paul H P via R-sig-meta-analysis
>>> Sent: Thursday, 13 April, 2023 16:53
>>> To: r-sig-meta-analysis using r-project.org
>>> Cc: Hanel, Paul H P
>>> Subject: [R-meta] R-square (change) as effect size
>>>
>>> Hello,
>>>
>>> Is it possible to run a meta-analysis using R-square (or R) as an effect size
>in
>>> the same way as you run a meta-analysis using Pearson's r?
>>>
>>> Specifically, I am interested in which of two sets of psychological constructs
>is
>>> better in predicting a range of outcomes such as well-being or self-reported
>>> behaviour. Set 1 consists of five variables (the so-called Big-5 personality
>>> traits) whereas set 2 consists of 10 variables (Schwartz's 10 value types). I
>>> have the adjusted R-squares from linear regressions for both sets of variables
>as
>>> well as the adjusted R-square change. For example, does set 1 explain more
>>> variance in well-being than set 2?
>>>
>>> Thanks
>>> Paul