<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal-compose;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoPlainText">PS: Just realised the table doesn't look like it should be. So I copied in a screenshot of it and uploaded it to a Google Doc, just in case
<a href="https://docs.google.com/document/d/1xICNhHQ_Te_riYjozG90d6idqvc8-PSSk7eLZkzG2ls/edit?usp=sharing">
https://docs.google.com/document/d/1xICNhHQ_Te_riYjozG90d6idqvc8-PSSk7eLZkzG2ls/edit?usp=sharing</a>
<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><img border="0" width="937" height="191" style="width:9.7583in;height:1.9916in" id="Picture_x0020_1" src="cid:image001.png@01D972FA.B3D9B4C0"><o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><span lang="EN-US" style="mso-fareast-language:EN-GB">-----Original Message-----<br>
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces@r-project.org> On Behalf Of Hanel, Paul H P via R-sig-meta-analysis<br>
Sent: 19 April 2023 20:04<br>
To: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer@maastrichtuniversity.nl>; R Special Interest Group for Meta-Analysis <r-sig-meta-analysis@r-project.org><br>
Cc: Hanel, Paul H P <p.hanel@essex.ac.uk><br>
Subject: Re: [R-meta] R-square (change) as effect size</span><o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Hi Wolfgang,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">I have the adjusted R-square and adjusted R-square change values for both sets of predictors and each DV. Luckily, many researchers were forthcoming and shared their raw data, since only a few papers reported the required hierarchical
regression results.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">See below for an example table (Moderators are omitted). It looks like personality traits (Big-5) explain more variance in well-being than human values, whereas values explain more variance in religiosity than traits.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Thank you,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Paul<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Paper ID<o:p></o:p></p>
<p class="MsoPlainText">Study ID<o:p></o:p></p>
<p class="MsoPlainText">DV<o:p></o:p></p>
<p class="MsoPlainText">N<o:p></o:p></p>
<p class="MsoPlainText">Rt2<o:p></o:p></p>
<p class="MsoPlainText">Rt2 (change)<o:p></o:p></p>
<p class="MsoPlainText">Rv2<o:p></o:p></p>
<p class="MsoPlainText">Rv2 (change)<o:p></o:p></p>
<p class="MsoPlainText">1<o:p></o:p></p>
<p class="MsoPlainText">1<o:p></o:p></p>
<p class="MsoPlainText">Religiosity<o:p></o:p></p>
<p class="MsoPlainText">987<o:p></o:p></p>
<p class="MsoPlainText">.05<o:p></o:p></p>
<p class="MsoPlainText">.02<o:p></o:p></p>
<p class="MsoPlainText">.15<o:p></o:p></p>
<p class="MsoPlainText">.12<o:p></o:p></p>
<p class="MsoPlainText">1<o:p></o:p></p>
<p class="MsoPlainText">2<o:p></o:p></p>
<p class="MsoPlainText">Well-being<o:p></o:p></p>
<p class="MsoPlainText">789<o:p></o:p></p>
<p class="MsoPlainText">.20<o:p></o:p></p>
<p class="MsoPlainText">.15<o:p></o:p></p>
<p class="MsoPlainText">.10<o:p></o:p></p>
<p class="MsoPlainText">.05<o:p></o:p></p>
<p class="MsoPlainText">2<o:p></o:p></p>
<p class="MsoPlainText">1<o:p></o:p></p>
<p class="MsoPlainText">Religiosity<o:p></o:p></p>
<p class="MsoPlainText">654<o:p></o:p></p>
<p class="MsoPlainText">.05<o:p></o:p></p>
<p class="MsoPlainText">.02<o:p></o:p></p>
<p class="MsoPlainText">.15<o:p></o:p></p>
<p class="MsoPlainText">.12<o:p></o:p></p>
<p class="MsoPlainText">2<o:p></o:p></p>
<p class="MsoPlainText">2<o:p></o:p></p>
<p class="MsoPlainText">Well-being<o:p></o:p></p>
<p class="MsoPlainText">456<o:p></o:p></p>
<p class="MsoPlainText">.30<o:p></o:p></p>
<p class="MsoPlainText">.25<o:p></o:p></p>
<p class="MsoPlainText">.10<o:p></o:p></p>
<p class="MsoPlainText">.05<o:p></o:p></p>
<p class="MsoPlainText">Note. Rt2: Amount of variance traits explain in DV, Rt2 (change): Amount of variance traits explain beyond values in DV; Rv2: Amount of variance values explain in DV, Rv2 (change): Amount of variance values explain beyond values in DV.
All adjusted R2-values. Rt2 + Rv2 (change) <span style="font-family:"Tahoma",sans-serif">
�</span> Rv2 + Rt2 (change).<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><span lang="DE">-----Original Message-----<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="DE">From: Viechtbauer, Wolfgang (NP) <</span><a href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl"><span lang="DE" style="color:windowtext;text-decoration:none">wolfgang.viechtbauer@maastrichtuniversity.nl</span></a><span lang="DE">><o:p></o:p></span></p>
<p class="MsoPlainText">Sent: 18 April 2023 11:17<o:p></o:p></p>
<p class="MsoPlainText">To: Hanel, Paul H P <<a href="mailto:p.hanel@essex.ac.uk"><span style="color:windowtext;text-decoration:none">p.hanel@essex.ac.uk</span></a>>; R Special Interest Group for Meta-Analysis <<a href="mailto:r-sig-meta-analysis@r-project.org"><span style="color:windowtext;text-decoration:none">r-sig-meta-analysis@r-project.org</span></a>><o:p></o:p></p>
<p class="MsoPlainText">Subject: RE: [R-meta] R-square (change) as effect size<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">CAUTION: This email was sent from outside the University of Essex. Please do not click any links or open any attachments unless you recognise and trust the sender. If you are unsure whether the content of the email is safe or have any
other queries, please contact the IT Helpdesk.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Hi Paul,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">What kind of data do you actually have? Do you have the two adjusted R^2 values for the two models of interest for each study? Or do you have studies where some provide the adjusted R^2 for the first model and other studies that provide
the adjusted R^2 for the other model?<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">> Would it be possible to transform the adjusted R-square to make its<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">> distribution more normal, take its squareroot and then treat it as a<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">> correlation coefficient using rma()?<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">No, it's not that simple. Just to give a counter-example: Say you have a simple regression model of the form y = beta0 + beta1 x + e and take the R^2 value from that model. Then sqrt(R^2) is equal to the *absolute value* of the correlation
between x and y. Since it is an absolute value, it isn't going to behave like a regular/signed correlation coefficient and cannot be treated as such.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Best,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Wolfgang<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>-----Original Message-----<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>From: Hanel, Paul H P [<a href="mailto:p.hanel@essex.ac.uk"><span style="color:windowtext;text-decoration:none">mailto:p.hanel@essex.ac.uk</span></a>]<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Sent: Tuesday, 18 April, 2023 12:00<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>To: R Special Interest Group for Meta-Analysis; Michael Dewey<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Cc: Viechtbauer, Wolfgang (NP)<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Subject: RE: [R-meta] R-square (change) as effect size<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Dear Wolfgang and Michael,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Thank you. After also having had a look at the links you provided, I am<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>not sure whether it would be best to focus on the adjusted R-square<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>change or the adjusted R-squares. Focusing on the adjusted R-square<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>values seems to make slightly more sense to me, but either should work<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>for what I have in mind. Would it be possible to transform the adjusted<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>R-square to make its distribution more normal, take its squareroot and then treat it as a correlation coefficient using rma()?<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Regarding the third point, the 'directionless' of R-square: This is not<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>overly relevant to my research question. I am interested in whether a<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>set of five predictors (personality traits, Big-5) explain more or less<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>variance in a range of outcome variable than a set of ten predictors<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>(human values). If a specific personality trait or value is correlated<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>with the outcome variables is something for future research - or has already been done.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Best,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Paul<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>-----Original Message-----<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>From: R-sig-meta-analysis <<a href="mailto:r-sig-meta-analysis-bounces@r-project.org%3cmailto:r-sig-meta-analysis-bounces@r-project.org"><span style="color:windowtext;text-decoration:none">r-sig-meta-analysis-bounces@r-project.org<mailto:r-sig-meta-analysis-bounces@r-project.org</span></a>>><o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>On Behalf Of Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Sent: 18 April 2023 09:26<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>To: Michael Dewey <<a href="mailto:lists@dewey.myzen.co.uk%3cmailto:lists@dewey.myzen.co.uk"><span style="color:windowtext;text-decoration:none">lists@dewey.myzen.co.uk<mailto:lists@dewey.myzen.co.uk</span></a>>>; R Special Interest
Group<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>for Meta- Analysis <<a href="mailto:r-sig-meta-analysis@r-project.org%3cmailto:r-sig-meta-analysis@r-project.org"><span style="color:windowtext;text-decoration:none">r-sig-meta-analysis@r-project.org<mailto:r-sig-meta-analysis@r-project.org</span></a>>><o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><span lang="DE">>Cc: Viechtbauer, Wolfgang (NP)<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE">><</span><a href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl%3cmailto:wolfgang.viechtbauer@maastrichtuniversity.nl"><span lang="DE" style="color:windowtext;text-decoration:none">wolfgang.viechtbauer@maastrichtuniversity.nl<mailto:wolfgang.viechtbauer@maastrichtuniversity.nl</span></a><span lang="DE">>><o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText">>Subject: Re: [R-meta] R-square (change) as effect size<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Great, thanks. Also found this thread:<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><a href="https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-March/002708.ht"><span style="color:windowtext;text-decoration:none">https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-March/002708.ht</span></a><o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>ml<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>which goes a bit in the same direction.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>If one focuses on the difference between the two R^2 values, then this<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>might hold some promise, but there are still these pesky little<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>technical details to figure out -- (approximate) normality of the<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>sampling distribution and the sampling variance of such a difference.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>P.S.: Just as a reminder to all, there is a custom Google search set up<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>for searching the mailing list archives here:<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><a href="https://cse.google.com/cse?cx=ee4b2e6c93b6a9667"><span style="color:windowtext;text-decoration:none">https://cse.google.com/cse?cx=ee4b2e6c93b6a9667</span></a> One caveat: Google<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>crawls the archives only periodically, so recent posts will not show up<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>in the search (for example, I tried a search for "R2" and it doesn't<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>bring up the thread from January, but it did lead me to the one from 2021).<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Best,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>Wolfgang<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>-----Original Message-----<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>From: Michael Dewey [<a href="mailto:lists@dewey.myzen.co.uk"><span style="color:windowtext;text-decoration:none">mailto:lists@dewey.myzen.co.uk</span></a>]<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>Sent: Monday, 17 April, 2023 17:04<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>To: R Special Interest Group for Meta-Analysis<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>Cc: Viechtbauer, Wolfgang (NP)<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>Subject: Re: [R-meta] R-square (change) as effect size<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>Dear Wolfgang, you were looking for<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>><a href="https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-January/004331"><span style="color:windowtext;text-decoration:none">https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-January/004331</span></a>.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>html and the surrounding thread although my hint there only addressed<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><span lang="DE">>>you point 3.<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE">>><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE">>>Michael<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE">>><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE">>>On 17/04/2023 13:23, Viechtbauer, Wolfgang (NP) via<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText"><span lang="DE">>>R-sig-meta-analysis<o:p></o:p></span></p>
<p class="MsoPlainText"><span lang="DE"><o:p> </o:p></span></p>
<p class="MsoPlainText">>>wrote:<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> Dear Paul,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> I think the issue of using R^2 as an effect size measure for a<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> meta-analysis<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>has come up before on this mailing list, although I can't find the<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>threads right now. In any case, there are several practical issues:<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> 1) The sampling distribution of (adjusted) R^2 is not normal, so one<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> needs to<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>figure out some appropriate normalizing transformation.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> 2) One also needs to figure out the sampling variance of the<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> (transformed)<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>(adjusted) R^2 value.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> 3) One can also debate the usefulness of meta-analyzing a 'directionless'<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>measure such as (adjusted) R^2. If you focus on the difference in<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>(adjusted) R^2 though (of two non-nested models), then I think this<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>issue is less concerning (and the sampling distribution of such a<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>difference might actually be somewhat normal). However, this then<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>raises another problem: The two (transformed)<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>(adjusted) R^2 values are not independent if they come from the same<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>sample and so one now also needs to figure out their covariance. If<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>they do not come from the same sample, then this alleviates this<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>particular issue, but makes the evidence much weaker due to potential<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>confounding.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> Best,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">>>> Wolfgang<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"> [[alternative HTML version deleted]]<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
</div>
</body>
</html>