[R-meta] R-square (change) as effect size

[Viechtbauer, Wolfgang (NP)]

That is correct (unless you can figure out a way to compute the sampling variances and the covariance without the raw data).

Best,
Wolfgang

>-----Original Message-----
>From: Hanel, Paul H P [mailto:p.hanel using essex.ac.uk]
>Sent: Sunday, 23 April, 2023 20:16
>To: Viechtbauer, Wolfgang (NP); R Special Interest Group for Meta-Analysis
>Subject: RE: [R-meta] R-square (change) as effect size
>
>Hi Wolfgang,
>
>Thank you.
>
>Sorry, I should have been clearer. I have the raw data for perhaps half of the
>studies.
>
>I take that a meta-analysis would only be possible for the studies for which I
>have the raw data using the method outlined below, but not for the studies for
>which I only have the adjusted R-squares and adjusted R-square changes?
>
>Best wishes,
>Paul
>
>-----Original Message-----
>From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: 20 April 2023 17:01
>To: Hanel, Paul H P <p.hanel using essex.ac.uk>; R Special Interest Group for Meta-
>Analysis <r-sig-meta-analysis using r-project.org>
>Subject: RE: [R-meta] R-square (change) as effect size
>
>Sidenote: Posting in plain-text will prevent tables from getting scrambled up.
>
>If you have the raw data, then this of course opens up many possibilities,
>especially for obtaining the sampling variances (and covariances) of whatever
>values you are interested in. In particular, one could use bootstrapping to
>obtain these values.
>
>In particular, for each study, you could compute y_1 = (Rt2 - Rv2) for
>religiosity and y_2 = (Rt2 - Rv2) for well-being in each bootstrap sample and
>then use var(y_1) and var(y_2) from these bootstrap replicates as the sampling
>variance of the two estimates and cov(y_1, y_2) as their covariance. It would
>also be good to examine the bootstrap distributions of y_1 and y_2 to see if they
>are somewhat normal.
>
>Then you can construct a dataset like this:
>
>study estimate dv
>1     y_1      religiosity
>1     y_2      well-being
>
>with the corresponding V matrix containing the 2x2 blocks with var(y_1), cov(y_1,
>y_2), and var(y_2).
>
>And now you are in the exact same situation as this:
>
>https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwviechtb.github.io%2fmetadat%
>2freference%2fdat.berkey1998.html&c=E,1,w6DcQI3Tp2q5TItSiACCatYNaeLukdqL9SBmZb879
>IY_lWR8NDfjK5dbJA3lFfU87sRlW-
>RIVr97ctb7JBu0hgn3oip1adl5nY6vYdEHmuHdRd_rsjEIQDbl&typo=1
>
>so you can use
>
>res <-
>https://linkprotect.cudasvc.com/url?a=https%3a%2f%2frma.mv&c=E,1,FxhB1OcOXdnxSGLw
>lT5Twnc73lbSgCIJF4GWiKLOOhov8809880vgxZXr7yyePTUsFMkm72qgpHZHMuyEcqJkJZsZ3YG7Bu17
>qHplkaxlBmt&typo=1(estimate, V, mods = ~ dv, random = ~ dv | study, struct="UN",
>data=dat)
>
>to estimate the average difference between Rt2 and Rv2 for each DV and test if
>there is a difference between thenm.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Hanel, Paul H P [mailto:p.hanel using essex.ac.uk]
>>Sent: Wednesday, 19 April, 2023 21:11
>>To: R Special Interest Group for Meta-Analysis; Viechtbauer, Wolfgang
>>(NP)
>>Subject: RE: [R-meta] R-square (change) as effect size
>>
>>PS: Just realised the table doesn't look like it should be. So I copied
>>in a screenshot of it and uploaded it to a Google Doc, just in case
>>
>>https://docs.google.com/document/d/1xICNhHQ_Te_riYjozG90d6idqvc8-
>>PSSk7eLZkzG2ls/edit?usp=sharing
>>
>>-----Original Message-----
>>From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org>
>>On Behalf Of Hanel, Paul H P via R-sig-meta-analysis
>>Sent: 19 April 2023 20:04
>>To: Viechtbauer, Wolfgang (NP)
>><wolfgang.viechtbauer using maastrichtuniversity.nl>; R Special Interest
>>Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
>>Cc: Hanel, Paul H P <p.hanel using essex.ac.uk>
>>Subject: Re: [R-meta] R-square (change) as effect size
>>
>>Hi Wolfgang,
>>
>>I have the adjusted R-square and adjusted R-square change values for
>>both sets of predictors and each DV. Luckily, many researchers were
>>forthcoming and shared their raw data, since only a few papers reported
>>the required hierarchical regression results.
>>
>>See below for an example table (Moderators are omitted). It looks like
>>personality traits (Big-5) explain more variance in well-being than
>>human values, whereas values explain more variance in religiosity than traits.
>>
>>Thank you,
>>
>>Paul
>>
>>Paper ID Study ID DV          N     Rt2   Rt2 (change)   Rv2   Rv2 (change)
>>1        1        Religiosity 987   .05   .02            .15   .12
>>1        2        Well-being  789   .20   .15            .10   .05
>>2        1        Religiosity 654   .05   .02            .15   .12
>>2        2        Well-being  456   .30   .25            .10   .05
>>
>>Note. Rt2: Amount of variance traits explain in DV, Rt2 (change):
>>Amount of variance traits explain beyond values in DV; Rv2: Amount of
>>variance values explain in DV, Rv2 (change): Amount of variance values
>>explain beyond values in DV. All adjusted R2-values. Rt2 + Rv2 (change) =~ Rv2 +
>Rt2 (change).
>>
>>-----Original Message-----
>>From: Viechtbauer, Wolfgang (NP)
>><wolfgang.viechtbauer using maastrichtuniversity.nl>
>>Sent: 18 April 2023 11:17
>>To: Hanel, Paul H P <p.hanel using essex.ac.uk>; R Special Interest Group for
>>Meta- Analysis <r-sig-meta-analysis using r-project.org>
>>Subject: RE: [R-meta] R-square (change) as effect size
>>
>>Hi Paul,
>>
>>What kind of data do you actually have? Do you have the two adjusted
>>R^2 values for the two models of interest for each study? Or do you
>>have studies where some provide the adjusted R^2 for the first model
>>and other studies that provide the adjusted R^2 for the other model?
>>
>>> Would it be possible to transform the adjusted R-square to make its
>>> distribution more normal, take its squareroot and then treat it as a
>>> correlation coefficient using rma()?
>>
>>No, it's not that simple. Just to give a counter-example: Say you have
>>a simple regression model of the form y = beta0 + beta1 x + e and take
>>the R^2 value from that model. Then sqrt(R^2) is equal to the *absolute
>>value* of the correlation between x and y. Since it is an absolute
>>value, it isn't going to behave like a regular/signed correlation coefficient
>and cannot be treated as such.
>>
>>Best,
>>Wolfgang