[R-meta] [External] RE: Effect Sizes and Beta Coefficients

[Hall, Rebecca]

Dear Wolfgang,

Many thanks for your assistance so far. We are still looking at papers using various statistical analyses, and have come across one which utilises a one-way MANCOVA. Please could you advise on the validity of calculating an effect size from this type of analysis, as the reported F and p values are altered by controlling for covariates, whereas this has not been the case in other studies.

Similarly, we have a paper reporting odds ratios which, if we are correct, we can directly convert to an effect size (pearson's r). However, the odds ratios in question are presented as two models, one with more covariates than the other. Is there a particular model you would suggest including (see below)?

[cid:b83e5b01-73a0-4682-87bf-9d6482f24a44]

Best,
Rebecca

Dr Rebecca Hall | Research Associate
Department of Psychology | Lancaster University
[cid:e30394e5-20ff-40d4-8c5e-a52658e54d4c]   [cid:24b9e1f2-c402-4ef3-8abd-35627f2c4cbf]
________________________________
From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: 13 June 2024 13:43
To: Hall, Rebecca <r.hall5 using lancaster.ac.uk>; R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
Subject: RE: [External] RE: [R-meta] Effect Sizes and Beta Coefficients

Dear Rebecca,

Please see below for my answer.

Best,
Wolfgang

> -----Original Message-----
> From: Hall, Rebecca <r.hall5 using lancaster.ac.uk>
> Sent: Wednesday, June 12, 2024 17:56
> To: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>; R
> Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
> Subject: Re: [External] RE: [R-meta] Effect Sizes and Beta Coefficients
>
> Dear Wolfgang,
>
> Many thanks for your response. Am I correct in thinking that I can convert a p-
> value in R using qt(1- p-value, n-1)?

I assume you are asking about a p-value from a regression model. First of all, this is typically a two-sided p-value, so the given p-value must first be divided by 2. Also, the degrees of freedom are n-p-1 for a regression model with p regression coefficients. So it would be:

qt(1-pval/2, df=n-p-1)

Note that the (two-sided) p-value does not tell you anything about the sign of the regression coefficient (which in turn corresponds to the sign of the test statistic and hence the sign of the (semi)partial correlation), so additional information reported needs to be used to determine the correct sign.

> Further to this, please could you advise how I might then calculate the partial
> correlation from a one-sample t-test?

You can't. A one-sample t-test is used to test if the mean of a single variable differs significantly from some value. It is not possible to compute the partial correlation for a single variable.

> Best,
> Rebecca
>
> Dr Rebecca Hall | Research Associate
> Department of Psychology | Lancaster University
>
> ________________________________________
> From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Sent: 11 June 2024 17:21
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Cc: Hall, Rebecca <r.hall5 using lancaster.ac.uk>
> Subject: [External] RE: [R-meta] Effect Sizes and Beta Coefficients
>
> This email originated outside the University. Check before clicking links or
> attachments.
>
> Dear Rebecca,
>
> This is my personal opinion: I would consider this approach outdated.
>
> Typically, in a situation like this, one also knows the t-statistic for the
> coefficient of interest (or its p-value from which one can back-calculate the t-
> statistic). In that case, one can compute the (semi)partial correlation
> coefficient for the coefficient.
>
> However, whether one should combine such 'partial' effect sizes with bivariate
> correlations is debatable in the first place. A relevant article that
> essentially argues against this is:
>
> Aloe, A. M. (2015). Inaccuracy of regression results in replacing bivariate
> correlations. Research Synthesis Methods, 6(1), 21-27.
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1002%2Fjrsm.1126&data=05%7C02%7Challr4%40live.lancs.ac.uk%7C2ce13474f1ec470a8a9c08dc8ba66749%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C638538794019673955%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=OUXIjY07mR54gO5eHMgodzvyAuc3uB4CnWfESXhYNJA%3D&reserved=0<https://doi.org/10.1002/jrsm.1126>
>
> At least, one could try to capture some of the heterogeneity introduced by this
> by including a moderator in the model that indicates the type of correlation
> coefficient. With enough studies, one could even go a step further and include
> moderators that indicate which covariates were included in the original
> regression models from which the (semi)partial correlations were obtained (as a
> bunch of dummy variables).
>
> Best,
> Wolfgang
>
> > -----Original Message-----
> > From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On
> Behalf
> > Of Hall, Rebecca via R-sig-meta-analysis
> > Sent: Tuesday, June 11, 2024 17:57
> > To: r-sig-meta-analysis using r-project.org
> > Cc: Hall, Rebecca <r.hall5 using lancaster.ac.uk>
> > Subject: [R-meta] Effect Sizes and Beta Coefficients
> >
> > Dear all,
> >
> > I have a question regarding the use of a beta coefficient as a substitute for
> > effect size where a paper lacks statistical data for Pearson's r to otherwise
> be
> > calculated.
> >
> > Peterson & Brown (2005) support the use of standardised beta coefficients and
> > relative SE in the place of correlations, but Roth et al. (2018) criticise
> this.
> > I'm therefore wondering whether there is a general consensus regarding the use
> > of beta coefficients, and should Peterson & Brown's approach no longer be
> > appropriate then I would be gladly advised on the alternative method that
> should
> > be utilised.
> >
> > Many thanks,
> > Rebecca
> >
> > Dr Rebecca Hall | Research Associate
> > Department of Psychology | Lancaster University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20240617/d1da18d2/attachment-0001.html>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 231540 bytes
Desc: image.png
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20240617/d1da18d2/attachment-0003.png>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-vohmdobc.png
Type: image/png
Size: 14445 bytes
Desc: Outlook-vohmdobc.png
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20240617/d1da18d2/attachment-0004.png>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-zheuyefz.png
Type: image/png
Size: 11783 bytes
Desc: Outlook-zheuyefz.png
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20240617/d1da18d2/attachment-0005.png>