[R-meta] Meta-Analysis using different correlation coefficients
|uk@@z@@t@@|e|ow|cz @end|ng |rom un|-o@n@brueck@de
Mon Feb 21 16:44:27 CET 2022
the order of the sentences in my original reply could be confusing, so
just to clarify:
I meant that it is generally possible to transform effect sizes and many
meta-analysts choose this option. However, I wasn't suggesting that this
approach is better/equivalent/inferior. If you choose this option, then
point-biserial correlation is mathematically equivalent to Pearson
correlation. Hence, there would be no need to transform the correlation.
However, I've mentioned that there are arguments against transforming
effect sizes. I should have added that it also means that one could
argue that point-biserial correlation and Pearson correlation shouldn't
be considered together in one meta-analysis. In general, "when in doubt
follow Wolfgang's advice" is a good heuristic. In particular, if you
don't have a strong opinion on a certain topic.
There are many decisions that can be criticized during peer-review in
primary studies (why ANOVA and not multi-level models? why multi-level
models and not structural equation modelling?) and meta-analysis (e.g.,
dealing with dependent effect sizes, inclusion of regression
coefficients when sythesizing correlations, inclusion of studies without
weighting/post-stratification). While there are some decisions on which
most of us would agree there are also areas where the opinions may vary
(or: some reviewers may have strong opinions and other can be
indifferent). Transformation/inclusion of other effect sizes belongs to
the second category.
In your case, the safest solution is to restrict meta-analysis to only
one type of effect sizes (e.g. Pearson correlation). You coud cite the
article (Jacobs & Viechtbauer) to justify exclusion of other effect sizes.
However, if this means that say only 5 studies meet your inclusion
criteria, then one could argue that including/transforming all effect
sizes is justifiable (more information, moderator analyses etc). One
could conduct a moderator analysis (effect size type, see previous
message) in order to test the robustness of the results and address
potential concerns of reviewers/readers.
There are many meta-analysts, who would consider all effect sizes
irrespective of the number of available studies but it is risky as some
reviewers could criticize the lack of justification. Providing a
justification could help, but in (rare?) cases it will be rejected.
Institute for Psychology
Research methods, psychological assessment, and evaluation
49074 Osnabrück (Germany)
Am 21.02.2022 um 14:04 schrieb Lena Pollerhoff:
> Dear Lukasz,
> Thank you so much for answering, we really appreciate the effort and
> your time.
> We had also done some research, which lead us to the following thread
> from Wolfgang Viechtbauer:
> and further to his paper (Jacobs & Viechtbauer) regarding point-biserial
> and biserial correlations („Estimation of the biserial correlation and
> its sampling variance for use in meta-analysis“) from 2016
> Based on this we were a little worried about including point-biserial
> and Pearson’s product-moment correlations in the same meta-analysis
> without transforming the point-biserial coefficients? E.g., in the paper
> they are saying „Unlike the point-biserial correlation coefficient,
> biserial coefficients can therefore be integrated with product-moment
> correlation coefficients in the same meta-analysis“, which lead us to
> assume (contrary to your suggestion) that point-biserial and
> product-moment correlation coefficients should not be included in the
> same meta-analysis without further transformation? But the paper also
> exclusively treats cases where a variable was *artificially*
> dichotomized (and still analyzed with point-biserial correlation instead
> of biserial correlation).
> But you are suggesting that in case the point-biserial correlation is
> based on a naturally occurring binary variable (e.g., gender), we can
> integrate it with Pearson's poruduct-moment correlation in the same
> Thank you so much in advance and best wishes
>> Am 15.02.2022 um 08:03 schrieb Lukasz Stasielowicz
>> <lukasz.stasielowicz using uni-osnabrueck.de
>> <mailto:lukasz.stasielowicz using uni-osnabrueck.de>>:
>> Dear Lena,
>> since you haven't received a response yet, I will address some points:
>> It is important to define a target effect size for the meta-analysis,
>> e.g. Pearson correlation r. If other effect sizes are reported in the
>> primary studies then one could transform them or use the raw data to
>> compute the target effect size.
>> If Pearson correlation is the target effect size and you have
>> point-biserial correlations then no transformation is necessary
>> because it is mathematically equivalent to Pearson correlation.
>> Spearman correlation and Pearson correlation will not always lead to
>> similar values, as the latter is less likely to detect nonlinear
>> (monotonic) relationships. Therefore, transforming Spearman
>> correlations to Pearson correlations could ensure, that the effect
>> size values can be interpreted the same way. In this particular case
>> one could use arcsin formulas:
>> de Winter, J. C. F., Gosling, S. D., & Potter, J. (2016). Comparing
>> the Pearson and Spearman correlation coefficients across distributions
>> and sample sizes: A tutorial using simulations and empirical data.
>> Psychological Methods, 21(3), 273–290. doi.org/10.1037/met0000079
>> You'll find formulas for other transformations in textbooks and other
>> journal articles.
>> While many meta-analysts tend to use all available information and
>> apply all possible transformations (d --> r, Spearman --> Pearson, OR
>> --> r), there are some people, who would point out that under certain
>> circumstances not all transformations are valid (e.g., d --> r can
>> the distinction between control group and experimental group be
>> thought as a part of an underlying continuous distribution or is it a
>> natural dichotomy?).
>> However, some people would argue that every effect size is useful (in
>> particular, if the meta-analysis is small).
>> Sometimes meta-analysts conduct a moderator analysis and compare
>> converted effect sizes (e.g., d-->r) with effect sizes that didn't
>> require converting (e.g. r), in order to check the influence of
>> pooling different designs/effects.
>> Multiple effect sizes: If there are multiple studies with multiple
>> effect sizes then a three-level meta-analysis could be useful.
>> Alternatively, one could use cluster-robust variance estimation, e.g.
>> If there is only one sample with multiple effects then choosing one
>> effect size (e.g., the typical operationalization of the constructs,
>> the most valid operationalization) could be an option. Alternatively,
>> one could compute a mean effect size, as you have mentioned in your
>> I hope it answers at least some of your questions. Good luck with your
>> Lukasz Stasielowicz
>> Osnabrück University
>> Institute for Psychology
>> Research methods, psychological assessment, and evaluation
>> Seminarstraße 20
>> 49074 Osnabrück (Germany)
>> Am 09.02.2022 um 07:54 schrieb
>> r-sig-meta-analysis-request using r-project.org
>> <mailto:r-sig-meta-analysis-request using r-project.org>:
>>> Send R-sig-meta-analysis mailing list submissions to
>>> r-sig-meta-analysis using r-project.org
>>> <mailto:r-sig-meta-analysis using r-project.org>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> or, via email, send a message with subject or body 'help' to
>>> r-sig-meta-analysis-request using r-project.org
>>> You can reach the person managing the list at
>>> r-sig-meta-analysis-owner using r-project.org
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of R-sig-meta-analysis digest..."
>>> Today's Topics:
>>> 1. Meta-Analysis using different correlation coefficients
>>> (Lena Pollerhoff)
>>> 2. Meta-Analysis using different correlation coefficients
>>> (Lena Pollerhoff)
>>> 3. Comparison to baseline- remove intercept or keep? (Danielle Hiam)
>>> 4. questions on some functions in metafor and clubsandwich
>>> (Brendan Hutchinson)
>>> Message: 1
>>> Date: Tue, 8 Feb 2022 13:39:52 +0100
>>> From: Lena Pollerhoff <lena using pollerhoff.de>
>>> To: r-sig-meta-analysis using r-project.org
>>> Subject: [R-meta] Meta-Analysis using different correlation
>>> Message-ID: <AA0B9D07-6B89-4BF7-91F7-D32B5FDE031A using pollerhoff.de>
>>> Content-Type: text/plain; charset="utf-8"
>>> We are currently conducting a meta-analysis based on correlation
>>> We have received a huge amount of raw datasets, so that we are able
>>> to calculate effect sizes/correlations coefficients on our own for
>>> many datasets, and we have other correlations extracted from the
>>> original pubs. Therefore I have a couple of questions:
>>> 1. If one variable is dichotomous and the other variable is
>>> continuous but not normally distributed, what kind of coefficient
>>> should be calculated? We’d go for point-biserial if the variable is
>>> naturally dichotomous (not artificially dichotomized), and for
>>> biserial correlation if the dichotomous variable was artificially
>>> dichotomized, but are worried that both require normal distribution
>>> of the continuous variable?
>>> 2. We are wondering how to best integrate person’s product moment
>>> correlation coefficients (both continuous, normally distributed
>>> variables), (point-) biserial correlation coefficients (for 1
>>> (artificial) dichotomous and 1 continuous variable) and spearman rang
>>> correlation coefficients (for non-parametric, both continuous
>>> variables) in one meta-analysis? Just use the raw values? Or is it
>>> better to transform them in a homogenous way (I’ve read Fisher’s z
>>> makes less sense for anything else than Pearson’s r as a
>>> variance-stabilizing procedure?)? Can spearman rho be converted using
>>> fisher’s z transformation? I’ve also read that it is not advisable to
>>> include product-moment correlation and point-biserial correlation in
>>> one meta-analysis, is there a way to convert the point-biserial
>>> correlation to something that can be integrated with Pearson’s r and
>>> Spearman’s rho?
>>> 3. I have multiple effect sizes within one sample and I want to
>>> aggregate them, how do I define rho in the aggregate function from
>>> the metafor package? Is it possible to calculate rho based on the raw
>>> datasets? Or would it better to think in a conservative way and
>>> assume perfect redundancy (i.e., rho = 0.9)?
>>> Thanks in advance for your time and effort!
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> <mailto:R-sig-meta-analysis using r-project.org>
More information about the R-sig-meta-analysis