[R-meta] Meta-analysis of R^2 Values

[Hanel, Paul H P]

Dear Wolfgang,

Thank you for making it possible to meta-analyse R-square values. Based on your documentation I take you are referring to the standard R-square values, not the adjusted one?

By entering the number of predictors mi, is your function computing the adjusted R-square values? If so, how would you run a meta-analysis with the adjusted R-squares?

Thank you
Paul

PS: I am happy to share the dataset with R-square values I am currently working on as soon as its finalised.

  

-----Original Message-----
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf Of Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis
Sent: 01 June 2023 13:51
To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Subject: [R-meta] Meta-analysis of R^2 Values

CAUTION: This email was sent from outside the University of Essex. Please do not click any links or open any attachments unless you recognise and trust the sender. If you are unsure whether the content of the email is safe or have any other queries, please contact the IT Helpdesk.

Hi all,

On a number of occasions, the question has been raised on this mailing list whether it is possible to meta-analyze R^2 values (I have also received this question a number of times via email). See, for example:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-March/002708.html
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-January/004325.html
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-April/004554.html

In these discussions, valid concerns about this have been raised. For example, R^2 values are 'directionless' (in contrast to the more commonly used outcome measures used for meta-analyses, where positive and negative values can cancel each other out). The question is also how to compute the sampling variance of R^2 values and whether some kind of transformation may be needed (to normalize the sampling distribution).

I share (and raised some of) these concerns but I would also say that it is not inherently wrong to meta-analyze R^2 values. Therefore, after a bit of further reading, thinking, and running some simulations, I have now implemented measures "R2" and "ZR2" in escalc(). The former is for raw R^2 values, although it should be better to use the latter as it uses a variance-stabilizing transformation of R^2 that also has normalizing properties (similar to the well-known r-to-z transformation for raw correlation coefficients). You can find the documentation about this here:

https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwviechtb.github.io%2fmetafor%2freference%2fescalc.html&c=E,1,8kWme078V2m1tDsJnIJFm8JpFICruAsy1Ba9XWC4tfjo3oXutZyVFjyofuPjVR1oDRLI9rntzQrksZDNHhkRJVEGJciWDi6aI-S-wjVgFSnSEhXphwqh&typo=1

(if you search for 'R-squared', you will find the right place in this ever growing help page).

Some of the caveats / limitations are also mentioned there (e.g., the equations assume that we are in a multivariate normal setting and that the true R^2 values are non-zero).

If you want to try this out, first install the 'devel' version of metafor:

install.packages("remotes")
remotes::install_github("wviechtb/metafor")

and then this will work:

library(metafor)

dat <- dat.aloe2013

par(mfrow=c(2,1))

dat <- escalc(measure="R2", r2i=R2, mi=preds, ni=n, data=dat, slab=study) res <- rma(yi, vi, data=dat) res forest(res, header=TRUE, xlim=c(-0.6,1.4), alim=c(0,1), refline=coef(res), efac=2) title(expression(bold("Using Raw " * R^2 * " Values")))

dat <- escalc(measure="ZR2", r2i=R2, mi=preds, ni=n, data=dat, slab=study) res <- rma(yi, vi, data=dat) res pred <- predict(res, transf=transf.ztor2) pred forest(res, header=TRUE, xlim=c(-0.6,1.4), alim=c(0,1), transf=transf.ztor2, refline=pred$pred, efac=2) title(expression(bold("Using z-transformed " * R^2 * " Values (back-transformed)")))

I cannot say whether a meta-analysis of the R^2 values for this particular dataset is sensible. Just using it for illustration purposes.

If somebody has a dataset with R^2 values where they have a legitimate reason for such a meta-analysis, I would love to hear about it. Any feedback in general is of course welcome.

Best,
Wolfgang

_______________________________________________
R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org To manage your subscription to this mailing list, go to:
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis