[R-sig-ME] Calculating effect sizes of fixed effects in lmer

[Daniel Lüdecke]

Dear Amie,

as additional comment to what has been said so far, I'd like to point to this forum post, which describes why it is difficult to get effect sizes like eta squared etc. from mixed models: https://afex.singmann.science/forums/topic/compute-effect-sizes-for-mixed-objects#post-295

Standardized coefficients are one possibility to report some kind of "effect size". The most accurate way would be standardizing the data before fitting the model (in particular when interaction terms are involved). Although I agree that having the "raw", unstandardized coefficients may provide a more intuitive interpretation, standardizing is sometimes even required just due to problem when fitting the model (like convergence issues).

Beyond that, you can - always having the caveats (especially) for mixed models in mind! - compute effect sizes like eta squared etc., and standardized coefficients with different methods of standardizing (posthoc as described by Wolfgang, or "refitting" the model on standardized version of the data) with the "effectsize" package: https://cran.r-project.org/package=effectsize There is also a dedicated webpage: https://easystats.github.io/effectsize/

Furthermore, the package just recently implemented a function for "pseudo-standardization" of parameters in mixed models. This approach addresses the issue raised by Wolfgang that mixed models have different sources of variability, and thus sd(y) would not properly account for this.

Hope this helps.

Best wishes
Daniel

-----Ursprüngliche Nachricht-----
Von: R-sig-mixed-models <r-sig-mixed-models-bounces using r-project.org> Im Auftrag von FAIRS Amie
Gesendet: Donnerstag, 24. September 2020 17:01
An: James Pustejovsky <jepusto using gmail.com>
Cc: r-sig-mixed-models using r-project.org
Betreff: Re: [R-sig-ME] Calculating effect sizes of fixed effects in lmer

Dear James,

Thank you so much ! I’ll check out all the references and your R package.

Best,

Amie

------------------
Dr. Amie Fairs
Post-doctorant
Aix-Marseille Université
Laboratoire Parole et Langage (LPL) | CNRS UMR 7309 | 5 Avenue Pasteur | 13100 Aix-en-Provence
Email : amie.fairs using univ-amu.fr<mailto:amie.fairs using univ-amu.fr>

While I may send this email outside of typical working hours, I have no expectation to receive an email outside of your typical hours.

From: James Pustejovsky <jepusto using gmail.com>
Sent: 24 September 2020 16:58
To: FAIRS Amie <amie.FAIRS using univ-amu.fr>
Cc: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>; r-sig-mixed-models using r-project.org
Subject: Re: [R-sig-ME] Calculating effect sizes of fixed effects in lmer

Hi Amie,

I agree very much with Wolfgang's perspective that one would ideally use outcomes such that unstandardized effects can be interpreted directly. If one does have to fall back on standardized effect sizes, there's a further question of what metric to use. Researchers often jump immediately to standardized mean differences, but there are certainly other possibilities, such as log response ratios for outcomes that are measured on ratio scales.

All that said, there has been a fair amount of work on standardized mean difference effect sizes for certain types of research designs that would usually be analyzed with multi-level models. A sampling (including some of my own):

  *   Hedges, L. V. (2007). Effect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32(4), 341-370.
  *   Hedges, L. V. (2011). Effect sizes in three-level cluster-randomized experiments. Journal of Educational and Behavioral Statistics, 36(3), 346-380.
  *   Pustejovsky, J. E., Hedges, L. V., & Shadish, W. R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39(5), 368-393.
  *   Stapleton, L. M., Pituch, K. A., & Dion, E. (2015). Standardized effect size measures for mediation analysis in cluster-randomized trials. The Journal of Experimental Education, 83(4), 547-582.
  *   Feingold, A. (2009). Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychological Methods, 14(1), 43.
One of my students and I have also developed an R package for estimating standardized mean differences from multilevel models fitted with nlme::lme()
https://CRAN.R-project.org/package=lmeInfo
Kind Regards,
James

	[[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-models using r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

--

_____________________________________________________________________

Universitätsklinikum Hamburg-Eppendorf; Körperschaft des öffentlichen Rechts; Gerichtsstand: Hamburg | www.uke.de
Vorstandsmitglieder: Prof. Dr. Burkhard Göke (Vorsitzender), Joachim Prölß, Prof. Dr. Blanche Schwappach-Pignataro, Marya Verdel
_____________________________________________________________________

SAVE PAPER - THINK BEFORE PRINTING