[R-meta] Effect size calculation

[James Pustejovsky]

Hi Tharaka,

Please keep the listserv cc'd. Responses below.

James

On Thu, Dec 23, 2021 at 5:40 AM Tharaka S. Priyadarshana
<tharakas.priyadarshana using gmail.com> wrote:
>
> Dear Prof. James,
>
> Thank you so much for your response, it is really helpful.
>
> May I ask you two more questions related to this, please?
> In my meta-analysis, I measured Pearson’s correlation coefficient between species richness (X) and landscape heterogeneity (Y).
> I measured this for 59 studies directly from the raw data.
> I also have a few more studies (9) that I could not get the raw data, but those papers have presented the standardized regression coefficients from multiple regression models. (1) In this case, can I use those standardized regression coefficients as an estimate for Pearson’s correlation coefficient between X and Y?
> If yes, could you please let me know a reference that I can cite, please?
>

Generally, standardized regression coefficients are not equivalent to
bi-variate Pearson correlations. The degree of discrepancy depends on
which predictor variables are included in the regression model and how
much variation in the outcome is explained by those predictors. If the
regressions include strong predictors, then the standardized
regression coefficient can be quite different from the Pearson
correlation.

If you have access to raw data from many studies, could you estimate
regressions using equivalent (or at least similar) predictors to those
used in these nine studies? That might help to get a sense of how big
a difference there is between the standardized regression coefficient
and the Pearson correlation.

> (2) Then, I also have a few more studies (6) that I was able to measure "(semi-)partial correlation coefficient" as described here by Prof. Wolfgang, https://wviechtb.github.io/metadat/reference/dat.aloe2013.html
> In this case, can I use those "(semi-)partial correlation coefficients" as an estimate for Pearson’s correlation coefficient between X and Y?
>

The answer is more or less the same as for standardized regression
coefficients. The difference between the semi-partial correlation and
the bivariate correlation depends on the other predictors that are
partialed out. This also holds for semi-partial correlations based on
different sets of predictors--the semi-partial between X and Y
controlling for Z1 is not equivalent to the semi-partial between X and
Y controlling for Z1 and Z2 (or for Z1, Z2, and Z3). To handle this, I
think Aloe and Becker have recommended coding which sets of predictors
are partialed out of each semi-partial r and using these codes as
moderators in the meta-analysis. Perhaps you could do something
similar in your synthesis?

> Thank you once again for your kind help.
>
> With best wishes,
> Tharaka
>
>
>
>
>
>
> On Thu, Dec 23, 2021 at 7:12 AM James Pustejovsky <jepusto using gmail.com> wrote:
>>
>> Hi Tharaka,
>>
>> There are formulas in the literature for converting between a
>> standardized mean difference (of which Hedges' g is an estimate) and
>> the Pearson correlation, as well as from other test statistics (F, t,
>> X^2); See Jacobs & Viechtbauer (2017) and Pustejovsky (2014). However,
>> the formulas are only valid under specific distributional assumptions
>> about the variables involved (e.g., that a continuous predictor has
>> been artificially dichotomized, and that interest is in the
>> correlation between the continuous predictor) and the outcome. For the
>> study you attached, it's not at all clear why the authors would use
>> the Pearson correlation coefficient as the effect size measure. If I
>> understand correctly, they are interested in the effects of
>> contrasting treatment conditions (AES vs. control) so the assumptions
>> for converting from SMD to r would not really make sense here.
>>
>> James
>>
>> Jacobs, P., & Viechtbauer, W. (2017). Estimation of the biserial
>> correlation and its sampling variance for use in meta‐analysis.
>> Research synthesis methods, 8(2), 161-180.
>> Pustejovsky, J. E. (2014). Converting from d to r to z when the design
>> uses extreme groups, dichotomization, or experimental control.
>> Psychological methods, 19(1), 92.
>>
>> On Tue, Dec 21, 2021 at 5:32 AM Tharaka S. Priyadarshana
>> <tharakas.priyadarshana using gmail.com> wrote:
>> >
>> > Dear all,
>> >
>> > I recently came across this paper (kindly see the attached pdf), and I am a bit confused about how these authors have measured the effect size for the MA. They have measured Pearson’s correlation coefficient in two different ways (see section 2.3, page 3).
>> >
>> > (i). Hedges’ g has been transformed to a Pearson’s correlation coefficient.
>> > (ii). Or, Pearson’s r has been calculated from F, t, or χ2 data.
>> >
>> > Is this a correct way of measuring the effect size? if yes, how this can be done??
>> > Then, can we really pool the effect sizes calculated from these two methods for the final analysis???
>> >
>> > Thank you,
>> > Tharaka
>> >
>> >
>> > _______________________________________________
>> > R-sig-meta-analysis mailing list
>> > R-sig-meta-analysis using r-project.org
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis