[R-meta] Meta-analysis of R^2 Values

[Viechtbauer, Wolfgang (NP)]

Dear Paul,

Correct - this is for standard R-square values, not adjusted ones. The number of predictors (mi) is needed for computing the sampling variances of the R^2 values (or for converting F-statistics or the corresponding p-values to R^2).

However, this approach to meta-analyzing R^2 values assumes that the sample sizes of the studies are sufficiently large for the difference between the standard and adjusted R-square values to be negligible anyway.

Would be great to have a real dataset, maybe as a contribution to the 'metadat' package (https://wviechtb.github.io/metadat/).

Best,
Wolfgang

>-----Original Message-----
>From: Hanel, Paul H P [mailto:p.hanel using essex.ac.uk]
>Sent: Wednesday, 07 June, 2023 12:12
>To: R Special Interest Group for Meta-Analysis
>Cc: Viechtbauer, Wolfgang (NP)
>Subject: RE: Meta-analysis of R^2 Values
>
>Dear Wolfgang,
>
>Thank you for making it possible to meta-analyse R-square values. Based on your
>documentation I take you are referring to the standard R-square values, not the
>adjusted one?
>
>By entering the number of predictors mi, is your function computing the adjusted
>R-square values? If so, how would you run a meta-analysis with the adjusted R-
>squares?
>
>Thank you
>Paul
>
>PS: I am happy to share the dataset with R-square values I am currently working
>on as soon as its finalised.
>
>-----Original Message-----
>From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
>Of Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis
>Sent: 01 June 2023 13:51
>To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>project.org>
>Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Subject: [R-meta] Meta-analysis of R^2 Values
>
>Hi all,
>
>On a number of occasions, the question has been raised on this mailing list
>whether it is possible to meta-analyze R^2 values (I have also received this
>question a number of times via email). See, for example:
>
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-March/002708.html
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-January/004325.html
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-April/004554.html
>
>In these discussions, valid concerns about this have been raised. For example,
>R^2 values are 'directionless' (in contrast to the more commonly used outcome
>measures used for meta-analyses, where positive and negative values can cancel
>each other out). The question is also how to compute the sampling variance of R^2
>values and whether some kind of transformation may be needed (to normalize the
>sampling distribution).
>
>I share (and raised some of) these concerns but I would also say that it is not
>inherently wrong to meta-analyze R^2 values. Therefore, after a bit of further
>reading, thinking, and running some simulations, I have now implemented measures
>"R2" and "ZR2" in escalc(). The former is for raw R^2 values, although it should
>be better to use the latter as it uses a variance-stabilizing transformation of
>R^2 that also has normalizing properties (similar to the well-known r-to-z
>transformation for raw correlation coefficients). You can find the documentation
>about this here:
>
>https://wviechtb.github.io/metafor/reference/escalc.html
>
>(if you search for 'R-squared', you will find the right place in this ever
>growing help page).
>
>Some of the caveats / limitations are also mentioned there (e.g., the equations
>assume that we are in a multivariate normal setting and that the true R^2 values
>are non-zero).
>
>If you want to try this out, first install the 'devel' version of metafor:
>
>install.packages("remotes")
>remotes::install_github("wviechtb/metafor")
>
>and then this will work:
>
>library(metafor)
>
>dat <- dat.aloe2013
>
>par(mfrow=c(2,1))
>
>dat <- escalc(measure="R2", r2i=R2, mi=preds, ni=n, data=dat, slab=study) res <-
>rma(yi, vi, data=dat) res forest(res, header=TRUE, xlim=c(-0.6,1.4), alim=c(0,1),
>refline=coef(res), efac=2) title(expression(bold("Using Raw " * R^2 * "
>Values")))
>
>dat <- escalc(measure="ZR2", r2i=R2, mi=preds, ni=n, data=dat, slab=study) res <-
>rma(yi, vi, data=dat) res pred <- predict(res, transf=transf.ztor2) pred
>forest(res, header=TRUE, xlim=c(-0.6,1.4), alim=c(0,1), transf=transf.ztor2,
>refline=pred$pred, efac=2) title(expression(bold("Using z-transformed " * R^2 * "
>Values (back-transformed)")))
>
>I cannot say whether a meta-analysis of the R^2 values for this particular
>dataset is sensible. Just using it for illustration purposes.
>
>If somebody has a dataset with R^2 values where they have a legitimate reason for
>such a meta-analysis, I would love to hear about it. Any feedback in general is
>of course welcome.
>
>Best,
>Wolfgang