[R-meta] Meta-Analysis using different correlation coefficients

Lukasz Stasielowicz |uk@@z@@t@@|e|ow|cz @end|ng |rom un|-o@n@brueck@de
Mon Feb 21 16:44:27 CET 2022

Dear Lena,

the order of the sentences in my original reply could be confusing, so 
just to clarify:
I meant that it is generally possible to transform effect sizes and many 
meta-analysts choose this option. However, I wasn't suggesting that this 
approach is better/equivalent/inferior. If you choose this option, then 
point-biserial correlation is mathematically equivalent to Pearson 
correlation. Hence, there would be no need to transform the correlation.

However, I've mentioned that there are arguments against transforming 
effect sizes. I should have added that it also means that one could 
argue that point-biserial correlation and Pearson correlation shouldn't 
be considered together in one meta-analysis. In general, "when in doubt 
follow Wolfgang's advice" is a good heuristic. In particular, if you 
don't have a strong opinion on a certain topic.

There are many decisions that can be criticized during peer-review in 
primary studies (why ANOVA and not multi-level models? why multi-level 
models and not structural equation modelling?) and meta-analysis (e.g., 
dealing with dependent effect sizes, inclusion of regression 
coefficients when sythesizing correlations, inclusion of studies without 
weighting/post-stratification). While there are some decisions on which 
most of us would agree there are also areas where the opinions may vary 
(or: some reviewers may have strong opinions and other can be 
indifferent). Transformation/inclusion of other effect sizes belongs to 
the second category.

In your case, the safest solution is to restrict meta-analysis to only 
one type of effect sizes (e.g. Pearson correlation). You coud cite the 
article (Jacobs & Viechtbauer) to justify exclusion of other effect sizes.
However, if this means that say only 5 studies meet your inclusion 
criteria, then one could argue that including/transforming all effect 
sizes is justifiable (more information, moderator analyses etc). One 
could conduct a moderator analysis (effect size type, see previous 
message) in order to test the robustness of the results and address 
potential concerns of reviewers/readers.
There are many meta-analysts, who would consider all effect sizes 
irrespective of the number of available studies but it is risky as some 
reviewers could criticize the lack of justification. Providing a 
justification could help, but in (rare?) cases it will be rejected.

Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstraße 20
49074 Osnabrück (Germany)

Am 21.02.2022 um 14:04 schrieb Lena Pollerhoff:
> Dear Lukasz,
> Thank you so much for answering, we really appreciate the effort and 
> your time.
> We had also done some research, which lead us to the following thread 
> from Wolfgang Viechtbauer: 
> https://chat.stackexchange.com/rooms/60238/discussion-between-mark-white-and-wolfgang 
> <https://chat.stackexchange.com/rooms/60238/discussion-between-mark-white-and-wolfgang>, 
> and further to his paper (Jacobs & Viechtbauer) regarding point-biserial 
> and biserial correlations („Estimation of the biserial correlation and 
> its sampling variance for use in meta-analysis“) from 2016 
> (https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1218 
> <https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1218>).
> Based on this we were a little worried about including point-biserial 
> and Pearson’s product-moment correlations in the same meta-analysis 
> without transforming the point-biserial coefficients? E.g., in the paper 
> they are saying „Unlike the point-biserial correlation coefficient, 
> biserial coefficients can therefore be integrated with product-moment 
> correlation coefficients in the same meta-analysis“, which lead us to 
> assume (contrary to your suggestion) that point-biserial and 
> product-moment correlation coefficients should not be included in the 
> same meta-analysis without further transformation? But the paper also 
> exclusively treats cases where a variable was *artificially* 
> dichotomized (and still analyzed with point-biserial correlation instead 
> of biserial correlation).
> But you are suggesting that in case the point-biserial correlation is 
> based on a naturally occurring binary variable (e.g., gender), we can 
> integrate it with Pearson's poruduct-moment correlation in the same 
> analysis?
> Thank you so much in advance and best wishes
> Lena
>> Am 15.02.2022 um 08:03 schrieb Lukasz Stasielowicz 
>> <lukasz.stasielowicz using uni-osnabrueck.de 
>> <mailto:lukasz.stasielowicz using uni-osnabrueck.de>>:
>> Dear Lena,
>> since you haven't received a response yet, I will address some points:
>> It is important to define a target effect size for the meta-analysis, 
>> e.g. Pearson correlation r. If other effect sizes are reported in the 
>> primary studies then one could transform them or use the raw data to 
>> compute the target effect size.
>> If Pearson correlation is the target effect size and you have 
>> point-biserial correlations then no transformation is necessary 
>> because it is mathematically equivalent to Pearson correlation.
>> Spearman correlation and Pearson correlation will not always lead to 
>> similar values, as the latter is less likely to detect nonlinear 
>> (monotonic) relationships. Therefore, transforming Spearman 
>> correlations to Pearson correlations could ensure, that the effect 
>> size values can be interpreted the same way. In this particular case 
>> one could use arcsin formulas:
>> de Winter, J. C. F., Gosling, S. D., & Potter, J. (2016). Comparing 
>> the Pearson and Spearman correlation coefficients across distributions 
>> and sample sizes: A tutorial using simulations and empirical data. 
>> Psychological Methods, 21(3), 273–290. doi.org/10.1037/met0000079 
>> <http://doi.org/10.1037/met0000079>
>> You'll find formulas for other transformations in textbooks and other 
>> journal articles.
>> While many meta-analysts tend to use all available information and 
>> apply all possible transformations (d --> r, Spearman --> Pearson, OR 
>> --> r), there are some people, who would point out that under certain 
>> circumstances not all transformations are valid (e.g., d --> r  can 
>> the distinction between control group and experimental group be 
>> thought as a part of an underlying continuous distribution or is it a 
>> natural dichotomy?).
>> However, some people would argue that every effect size is useful (in 
>> particular, if the meta-analysis is small).
>> Sometimes meta-analysts conduct a moderator analysis and compare 
>> converted effect sizes (e.g., d-->r) with effect sizes that didn't 
>> require converting (e.g. r), in order to check the influence of 
>> pooling different designs/effects.
>> Multiple effect sizes: If there are multiple studies with multiple 
>> effect sizes then a three-level meta-analysis could be useful. 
>> Alternatively, one could use cluster-robust variance estimation, e.g. 
>> cran.r-project.org/web/packages/clubSandwich/vignettes/meta-analysis-with-CRVE.html 
>> <http://cran.r-project.org/web/packages/clubSandwich/vignettes/meta-analysis-with-CRVE.html>
>> If there is only one sample with multiple effects then choosing one 
>> effect size (e.g., the typical operationalization of the constructs, 
>> the most valid operationalization) could be an option. Alternatively, 
>> one could compute a mean effect size, as you have mentioned in your 
>> message.
>> I hope it answers at least some of your questions. Good luck with your 
>> project!
>> Best,
>> Lukasz
>> -- 
>> Lukasz Stasielowicz
>> Osnabrück University
>> Institute for Psychology
>> Research methods, psychological assessment, and evaluation
>> Seminarstraße 20
>> 49074 Osnabrück (Germany)
>> Am 09.02.2022 um 07:54 schrieb 
>> r-sig-meta-analysis-request using r-project.org 
>> <mailto:r-sig-meta-analysis-request using r-project.org>:
>>> Send R-sig-meta-analysis mailing list submissions to
>>> r-sig-meta-analysis using r-project.org 
>>> <mailto:r-sig-meta-analysis using r-project.org>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>> or, via email, send a message with subject or body 'help' to
>>> r-sig-meta-analysis-request using r-project.org
>>> You can reach the person managing the list at
>>> r-sig-meta-analysis-owner using r-project.org
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of R-sig-meta-analysis digest..."
>>> Today's Topics:
>>>    1. Meta-Analysis using different correlation coefficients
>>>       (Lena Pollerhoff)
>>>    2. Meta-Analysis using different correlation coefficients
>>>       (Lena Pollerhoff)
>>>    3. Comparison to baseline- remove intercept or keep? (Danielle Hiam)
>>>    4. questions on some functions in metafor and clubsandwich
>>>       (Brendan Hutchinson)
>>> ----------------------------------------------------------------------
>>> Message: 1
>>> Date: Tue, 8 Feb 2022 13:39:52 +0100
>>> From: Lena Pollerhoff <lena using pollerhoff.de>
>>> To: r-sig-meta-analysis using r-project.org
>>> Subject: [R-meta] Meta-Analysis using different correlation
>>> coefficients
>>> Message-ID: <AA0B9D07-6B89-4BF7-91F7-D32B5FDE031A using pollerhoff.de>
>>> Content-Type: text/plain; charset="utf-8"
>>> Hello,
>>> We are currently conducting a meta-analysis based on correlation 
>>> coefficients.
>>> We have received a huge amount of raw datasets, so that we are able 
>>> to calculate effect sizes/correlations coefficients on our own for 
>>> many datasets, and  we have other correlations extracted from the 
>>> original pubs. Therefore I have a couple of questions:
>>> 1. If one variable is dichotomous and the other variable is 
>>> continuous but not normally distributed, what kind of coefficient 
>>> should be calculated? We’d go for point-biserial if the variable is 
>>> naturally dichotomous (not artificially dichotomized), and for 
>>> biserial correlation if the dichotomous variable was artificially 
>>> dichotomized, but are worried that both require normal distribution 
>>> of the continuous variable?
>>> 2. We are wondering how to best integrate person’s product moment 
>>> correlation coefficients (both continuous, normally distributed 
>>> variables), (point-) biserial correlation coefficients (for 1 
>>> (artificial) dichotomous and 1 continuous variable) and spearman rang 
>>> correlation coefficients (for non-parametric, both continuous 
>>> variables) in one meta-analysis? Just use the raw values? Or is it 
>>> better to transform them in a homogenous way (I’ve read Fisher’s z 
>>> makes less sense for anything else than Pearson’s r as a 
>>> variance-stabilizing procedure?)? Can spearman rho be converted using 
>>> fisher’s z transformation? I’ve also read that it is not advisable to 
>>> include product-moment correlation and point-biserial correlation in 
>>> one meta-analysis, is there a way to convert the point-biserial 
>>> correlation to something that can be integrated with Pearson’s r and 
>>> Spearman’s rho?
>>> 3. I have multiple effect sizes within one sample and I want to 
>>> aggregate them, how do I define rho in the aggregate function from 
>>> the metafor package? Is it possible to calculate rho based on the raw 
>>> datasets? Or would it better to think in a conservative way and 
>>> assume perfect redundancy (i.e., rho = 0.9)?
>>> Thanks in advance for your time and effort!
>>> Best,
>>> Lena
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org 
>> <mailto:R-sig-meta-analysis using r-project.org>
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

More information about the R-sig-meta-analysis mailing list