[R-sig-ME] eta-squared in lmer

[Reinhold Kliegl]

You may want to report an adjusted R-square value and add a footnote
that this is an area of active development. For example, in Oberauer &
Kliegl (2006, p. 605, Eq. 2,. Journal of Memory and Language, p. ), we
used a formula from (McElree & Dosher, 1989, Journal of Experimental
Psychology: General) and added the following footnote:

"Footnote 3. The development of R2 statistics or equivalent indicators
of goodness of fit for multilevel regression models is an active field
of research (e.g., Roberts & Monaco, 2006). For example, one problem
with the formula employed in this article is that the inclusion of
level-2 predictors (i.e., subject-level predictors) could
theoretically render the adjusted R2 statistic negative. We included
this statistic only for reasons of rough comparability of these
results with related earlier research and as an additional index for
the comparison of nested models involving different level-1 predictors
(i.e., item-level predictors)."

There is probably more recent work on this. At a general level, if
your theory expects a small effect, you should not be forced to
document a large one. Indeed, that might suggest that something went
wrong. Large effects are very desirable in applied settings (where you
do not care why an instrument works), but I do not think it is a
useful general criterion in the context of theory-guided research.

Reinhold Kliegl

On Tue, Apr 19, 2011 at 9:38 AM, Petar Milin <pmilin at ff.uns.ac.rs> wrote:
> Fine, I like logLik too, moreover, I am used to aic, bic and logLik, but it
> is hard to persuade stubborn reviewers and to convert journal policy. So,
> what can I do? How to make all happy? Furthermore, my model does have slope
> adjustment (a very cute one)! Hence, no ez-tool, no mcmc, ...
>
> Thanks!
> PM
>
> On 18/04/11 17:39, Mike Lawrence wrote:
>>
>> Actually, the ez package provides estimates of generalized eta-squared
>> for ANOVA only, sorry. That said, it could be argued that measures of
>> effect size (like eta-squared) represent a half-hearted attempt by
>> scientists to move beyond the impoverished information provided by
>> p-values, and that a full-hearted alternative that provides the
>> information we really want as scientists is the likelihood ratio. To
>> this end, ez *does* provide likelihood ratios for fixed effects in
>> mixed effects models with simple random effects structures (eg. no
>> "varying slopes" models).
>>
>>
>> On Mon, Apr 18, 2011 at 11:45 AM, Petar Milin<pmilin at ff.uns.ac.rs>  wrote:
>>>
>>> Hello ALL!
>>> Until now I have never been asked to provide (partial) eta-squared for
>>> any
>>> of my lmer models. However, reviewers in a journal where I plan to submit
>>> have strict policy of having this particular statistics. I know about EZ
>>> package, and that it could/should do something of the kind, but I do not
>>> know details. Please, can anyone tell me how can I get etas from typical
>>> lmer model? I know general logic of it, but never dig deeper. Moreover, I
>>> think I've read somewhere that it is biased. Can you suggest some good
>>> reference?
>>>
>>> Thanks,
>>> PM
>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>
> --
> #####################################################################################
> Petar Milin
> Department of Psychology, University of Novi Sad, Serbia
>  &
> Laboratory for Experimental Psychology, University of Belgrade, Serbia
>
> Address:   Dr Zorana Dindica 2, Novi Sad 21000, Serbia
> E-mail:    pmilin<AT>  ff.uns.ac.rs
> Tel.&Fax:  +381 21 458 948
>
> Official homepage (in Serbian):
> http://www.ff.uns.ac.rs/fakultet/ljudi/fakultet_odseci_psihologija_petar_milin.html
>
> Personal homepage:
> http://sites.google.com/site/pmilin/
> #####################################################################################
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>