[R-meta] [External] RE: Effect Sizes and Beta Coefficients

[Viechtbauer, Wolfgang (NP)]

Adding a bit to this:

With respect to the MANCOVA: What kind of 'effect size' are you trying to compute for this? This thread revolved around ((semi)partial) correlation coefficients, so I assume this is what you are trying to compute here. I am not sure what information you have from the MANCOVA, but generally I would say that it will be next to impossible to compute a correlation coefficient from such an analysis. Test statistics from a MANCOVA will not tell you about directionality and are a complex amalgamation of the group differences for multiple variables (adjusted for covariates). Things already get tricky if we take away the 'multivariate' part and take away the 'covariate' part, in which case we are left with an independent samples t-test. Translating the results from such a test into a type of correlation coefficient that can meaningfully be combined with the results from studies that measured both variables of interest on a continuous scale (and then report of regular Pearson product-moment correlation coefficient) requires assuming that the dichotomous variable that defines the two groups for the t-test is a dichotomized version of a continuous variable in which case one can compute a biserial correlation coefficient. Trying to do something like this based on an MANCOVA seems dubious at best.

As for your second question: As far as I can tell, the table you show does not appear to provide odds ratios, but the results from linear regression models (leaving aside that the table appears to be messed up, at least with respect to the last three columns). The values shown are beta coefficients (since outcome and predictors were z-scored -- which makes the B and beta columns redundant). If you want a partial correlation coefficient here, you can just compute B/SE to get the test statistic. Together with the number of predictors and the total sample size, one can then compute the partial correlation coefficient from this information. But I cannot tell you which of the models you should use for this (this is a substantive question).

Best,
Wolfgang

> -----Original Message-----
> From: Michael Dewey <lists using dewey.myzen.co.uk>
> Sent: Monday, June 17, 2024 17:13
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>; Viechtbauer, Wolfgang (NP)
> <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Cc: Hall, Rebecca <r.hall5 using lancaster.ac.uk>
> Subject: Re: [R-meta] [External] RE: Effect Sizes and Beta Coefficients
>
> Dear Rebecca
>
> In general estimates adjusted for different sets of covariates are not
> directly comparable. If the sets are almost the same then you may be
> able to argue for including them. It might also depend on strategic
> questions like how many studies you already have and whether adding one
> more is worth the hassle of explaining why you included it.
>
> In short as so often the answer is, how brave do you feel?
>
> Michael
>
> On 17/06/2024 15:27, Hall, Rebecca via R-sig-meta-analysis wrote:
> > Dear Wolfgang,
> >
> > Many thanks for your assistance so far. We are still looking at papers
> > using various statistical analyses, and have come across one which
> > utilises a one-way MANCOVA. Please could you advise on the validity of
> > calculating an effect size from this type of analysis, as the reported F
> > and p values are altered by controlling for covariates, whereas this has
> > not been the case in other studies.
> >
> > Similarly, we have a paper reporting odds ratios which, if we are
> > correct, we can directly convert to an effect size (pearson's r).
> > However, the odds ratios in question are presented as two models, one
> > with more covariates than the other. Is there a particular model you
> > would suggest including (see below)?
> >
> > Best,
> > Rebecca
> >
> > *Dr Rebecca Hall | Research Associate*
> > Department of Psychology | Lancaster University