[R-meta] Can z-transformed R-squared be used as an effect size?

[Michael Dewey]

Dear Matthew

I share your concerns and doubts about this. I would hesitate to go out 
on a limb over it but if you are reviewing it for a journal I would 
suggest politely telling the authors that you find it hard to justify 
and ask them to provide a reference or other wise justify it.

I do not make confidential comments to the editors on principle but if I 
did I would tell them that unless the authors can justify it the paper 
demands instant rejection.

Michael

On 17/03/2022 17:47, Matthew Yates wrote:
> Hello SIG-meta folks,
> 
> I have (what I think is) a pretty quick question. I'm currently conducting a peer-review of a meta-analysis in my field.
> 
> The authors of this manuscript elected to use fisher-Z-transformed R-squared values (note: NOT Pearson correlation coefficients, but their squared-values) as their 'effect size' statistic, and then calculated variance for the z-transformed R-squared values as for a typical Pearson correlation coefficient (r).
> 
> Is this a valid statistical effect size? The z-transformation, as I understand it, was developed specifically for Pearson correlation coefficients, so this strikes me as potentially problematic - the z-transformation, itself, is meant for variables that can span -1 to 1, so the underlying distribution of the transformed variables (0 to 1 for r-squared values) are inherently different. Similarly, estimating the variances based on the sample size (n) of the z-transformed R-squared values again strikes me as potentially problematic as well.
> 
> As far as I can tell, I think pretty much all of the studies being analysed were bivariate linear regressions, so there isn't an issue with non-linear relationships, covariates, etc (I saw another post on here asking that question....). I've just never seen this done before in a meta-analysis, or read of it in any literature, guides, etc. on how to conduct a meta-analysis. Most people typically just use the z-transformed Pearson correlation coefficients, rather than the R-squared values!
> 
> I've done a few meta-analyses myself, so am familiar with general techniques but would not consider myself an expert/specialist (most of mine were pretty basic). However, this strikes me as potentially problematic, and I was wondering what others with more statistical expertise in meta-analytic techniques might think of this issue.
> 
> Any input would be appreciated.
> 
> Thanks,
> 
> 
> Dr. Matthew C. Yates
> 
> Post-doctoral Researcher
> 
> Great Lakes Institute for Environmental Research (GLIER)
> University of Windsor
> 2990 Riverside Dr W,
> Windsor, ON N9C 1A2
> 
> (514) 919 5613
> 
> Website: https://matthewyates6.wixsite.com/ecologist
> 
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 

-- 
Michael
http://www.dewey.myzen.co.uk/home.html