[R-meta] R-square (change) as effect size

[Hanel, Paul H P]

Dear Wolfgang and Michael,

Thank you. After also having had a look at the links you provided, I am not sure whether it would be best to focus on the adjusted R-square change or the adjusted R-squares. Focusing on the adjusted R-square values seems to make slightly more sense to me, but either should work for what I have in mind. Would it be possible to transform the adjusted R-square to make its distribution more normal, take its squareroot and then treat it as a correlation coefficient using rma()?

Regarding the third point, the 'directionless' of R-square: This is not overly relevant to my research question. I am interested in whether a set of five predictors (personality traits, Big-5) explain more or less variance in a range of outcome variable than a set of ten predictors (human values). If a specific personality trait or value is correlated with the outcome variables is something for future research - or has already been done. 

Best,
Paul




-----Original Message-----
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf Of Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis
Sent: 18 April 2023 09:26
To: Michael Dewey <lists using dewey.myzen.co.uk>; R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Subject: Re: [R-meta] R-square (change) as effect size

CAUTION: This email was sent from outside the University of Essex. Please do not click any links or open any attachments unless you recognise and trust the sender. If you are unsure whether the content of the email is safe or have any other queries, please contact the IT Helpdesk.

Great, thanks. Also found this thread:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-March/002708.html

which goes a bit in the same direction.

If one focuses on the difference between the two R^2 values, then this might hold some promise, but there are still these pesky little technical details to figure out -- (approximate) normality of the sampling distribution and the sampling variance of such a difference.

P.S.: Just as a reminder to all, there is a custom Google search set up for searching the mailing list archives here: https://cse.google.com/cse?cx=ee4b2e6c93b6a9667 One caveat: Google crawls the archives only periodically, so recent posts will not show up in the search (for example, I tried a search for "R2" and it doesn't bring up the thread from January, but it did lead me to the one from 2021).

Best,
Wolfgang

>-----Original Message-----
>From: Michael Dewey [mailto:lists using dewey.myzen.co.uk]
>Sent: Monday, 17 April, 2023 17:04
>To: R Special Interest Group for Meta-Analysis
>Cc: Viechtbauer, Wolfgang (NP)
>Subject: Re: [R-meta] R-square (change) as effect size
>
>Dear Wolfgang, you were looking for
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-January/004331.
>html and the surrounding thread although my hint there only addressed 
>you point 3.
>
>Michael
>
>On 17/04/2023 13:23, Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis
>wrote:
>> Dear Paul,
>>
>> I think the issue of using R^2 as an effect size measure for a 
>> meta-analysis
>has come up before on this mailing list, although I can't find the 
>threads right now. In any case, there are several practical issues:
>>
>> 1) The sampling distribution of (adjusted) R^2 is not normal, so one 
>> needs to
>figure out some appropriate normalizing transformation.
>>
>> 2) One also needs to figure out the sampling variance of the 
>> (transformed)
>(adjusted) R^2 value.
>>
>> 3) One can also debate the usefulness of meta-analyzing a 'directionless'
>measure such as (adjusted) R^2. If you focus on the difference in 
>(adjusted) R^2 though (of two non-nested models), then I think this 
>issue is less concerning (and the sampling distribution of such a 
>difference might actually be somewhat normal). However, this then 
>raises another problem: The two (transformed)
>(adjusted) R^2 values are not independent if they come from the same 
>sample and so one now also needs to figure out their covariance. If 
>they do not come from the same sample, then this alleviates this 
>particular issue, but makes the evidence much weaker due to potential confounding.
>>
>> Best,
>> Wolfgang
>>
>>> -----Original Message-----
>>> From: R-sig-meta-analysis 
>>> [mailto:r-sig-meta-analysis-bounces using r-project.org]
>On
>>> Behalf Of Hanel, Paul H P via R-sig-meta-analysis
>>> Sent: Thursday, 13 April, 2023 16:53
>>> To: r-sig-meta-analysis using r-project.org
>>> Cc: Hanel, Paul H P
>>> Subject: [R-meta] R-square (change) as effect size
>>>
>>> Hello,
>>>
>>> Is it possible to run a meta-analysis using R-square (or R) as an 
>>> effect size
>in
>>> the same way as you run a meta-analysis using Pearson's r?
>>>
>>> Specifically, I am interested in which of two sets of psychological 
>>> constructs
>is
>>> better in predicting a range of outcomes such as well-being or 
>>> self-reported behaviour. Set 1 consists of five variables (the 
>>> so-called Big-5 personality
>>> traits) whereas set 2 consists of 10 variables (Schwartz's 10 value 
>>> types). I have the adjusted R-squares from linear regressions for 
>>> both sets of variables
>as
>>> well as the adjusted R-square change. For example, does set 1 
>>> explain more variance in well-being than set 2?
>>>
>>> Thanks
>>> Paul
_______________________________________________
R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org To manage your subscription to this mailing list, go to:
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis