Thanks for these pertinent comments.

I can't comment on the motivation for the original post. I have always felt
that a single dimensionless Rsq was fairly meaningless in the context of
mixed models.

Gelman & Pardoe's formula summarizes the fit at each level in the model
separately. This has more intuitive appeal, especially since I tend to fit
models containing fixed effects at the group level. The motivation then
would be to write a sentence along the lines of 'gender explains 5% of the
among-subject variance in orthodontic growth curves; age explains 80% of the
within-subject variation'.

Incidentally, G&P also state that negative Rsqs might be expected (for their
index): essentially it means that adding a fixed effect causes the variance
of a random effect to increase..

Best wishes, Nick


On 26 February 2010 14:30, Douglas Bates <bates@stat.wisc.edu> wrote:

> On Fri, Feb 26, 2010 at 7:37 AM, Nick Isaac <njbisaac@googlemail.com>
> wrote:
> > Sorry to be joining this late.
>
> > I have written some code to implement Gelman & Pardoe's Rsq for an lmer
> > object. It gives some believable results, but it's difficult to be
> confident
> > because of the translation from Bayesian into frequentist paradigms.
>
> > If anyone is interested then I'd be really happy to discuss this off-list
> > and share/develop the code.
>
> Assuming that one wants to define an R^2 measure, I think an argument
> could be made for treating the penalized residual sum of squares from
> a linear mixed model in the same way that we consider the residual sum
> of squares from a linear model.  Or one could use just the residual
> sum of squares without the penalty or the minimum residual sum of
> squares obtainable from a given set of terms, which corresponds to an
> infinite precision matrix.  I don't know, really.  It depends on what
> you are trying to characterize.
>
> In other words, what's the purpose?  What aspect of the R^2 for a
> linear model are you trying to generalize?
>
> I'm sorry if I sound argumentative but discussions like this sometimes
> frustrate me.  A linear mixed model does not behave exactly like a
> linear model without random effects so a measure that may be
> appropriate for the linear model does not necessarily generalize.  I'm
> not saying that this is the case but if the request is "I don't care
> what the number means or if indeed it means anything at all, just give
> me a number I can report", that's not the style of statistics I
> practice.
>
> I regard Bill Venables' wonderful unpublished paper "Exegeses on
> Linear Models" (just put the name in a search engine to find a copy -
> there is only one paper with "Exegeses" and "Linear Models" in the
> title) as required reading for statisticians.  As Bill emphasizes in
> that paper, statistics is not just a collection of formulas (many of
> which are based on approximations).  It's about models and comparing
> how well different models fit the observed data.  If we start with a
> formula and only ask ourselves "How do we generalize this formula?"
> we're missing the point.  We should start at the model.
>
> In a linear model the R^2 statistic is a dimensionless comparison of
> the quality of the current model fit, as measured by the residual sum
> of squares, to the fit one would obtain from a trivial model.  When
> the current model can be shown to contain a model with an intercept
> term only (and whose coefficient will be estimated by the mean
> response) then that model fit is the trivial model.  Otherwise the
> trivial model is a prediction of zero for each response.  We know that
> the trivial model will produce a greater residual sum of squares than
> the current model fit because the models are nested.  The R^2 is the
> proportion of variability not accounted for by the trivial model but
> accounted for by the current model (my apologies to my grammar
> teachers for having juxtaposed prepositions).
>
> The interesting point there is that when you think of the
> relationships between models you can determine how you handle the case
> of a model that does not have an intercept term.  If you start from
> the formula instead you can end up calculating a negative R^2 because
> you compare models that are not nested.  Such nonsensical results are
> often reported.  (I think it was the Mathematica documentation that
> gave a careful explanation of why you get a negative R^2 instead of
> recognizing that the formula they were using did not apply in certain
> cases.)
>
> It may be that there is a sensible measure of the quality of fit from
> a linear mixed model that generalizes the R^2 from a linear model.  I
> don't see an obvious candidate but I will freely admit that I haven't
> thought much about the problem.  I would ask others who are thinking
> about this to consider both the "what" and the "why".  George
> Mallory's justification of "because it's there" for attempting to
> climb Everest is perhaps a good justification for such endeavors
> (Mallory may have questioned his rationale as he lay freezing to death
> on the mountain).  I don't think it is a good justification for
> manipulating formulas.
>
>
> >
> > Best wishes, Nick
> >
> >
> > On 18 February 2010 06:55, Luisa Carvalheiro <lgcarvalheiro@gmail.com
> >wrote:
> >
> >> Hi Steve,
> >>
> >> Thanlks for reply and literature list. Here are 3 papers on  R2
> >> calculations for Mixed Models:
> >>
> >> M. Mittlbock, T. Waldhor. Adjustments for R2-measures for Poisson
> >> regression models. Computational Statistics & Data Analysis 34 (2000)
> >> 461-472
> >>
> >> M. Mittlbock Calculating adjusted R2 measures for Poisson regression
> >> Models. Computer Methods and Programs in Biomedicine 68 (2002) 205–214
> >>
> >> H. Liu,Y. Zheng and J. Shen. Goodness-of-fit measures of R2 for
> >> repeated measures mixed effect models Journal of Applied Statistics.
> >> 35, 2008, 1081–1092
> >>
> >>
> >> On Thu, Feb 18, 2010 at 1:46 AM, Steven J. Pierce <pierces1@msu.edu>
> >> wrote:
> >> > Luisa,
> >> >
> >> > I'm not aware of any packages for that, but I'd like the full citation
> >> for
> >> > the paper you mentioned. In exchange, here are some citations for
> other
> >> > papers about R-square measures in multilevel models that I've found.
> >> >
> >> > Edwards, L. J., Muller, K. E., Wolfinger, R. D., Qaqish, B. F., &
> >> > Schabenberger, O. (2008). An R2 statistic for fixed effects in the
> linear
> >> > mixed model. Statistics in Medicine, 27(29), 6137-6157. doi:
> >> > 10.1002/sim.3429
> >> >
> >> > Gelman, A., & Pardoe, I. (2006). Bayesian measures of explained
> variance
> >> and
> >> > pooling in multilevel (hierarchical) models. Technometrics, 48(2),
> >> 241-251.
> >> > doi: 10.1198/004017005000000517
> >> >
> >> > Kramer, M. (2005). R2 statistics for mixed models. Proceedings of the
> >> > Conference on Applied Statistics in Agriculture, 17, 148-160.
> Retrieved
> >> from
> >> >
> >>
> http://www.ars.usda.gov/sp2UserFiles/ad_hoc/12000000SpatialWorkshop/19Kramer
> >> > SupplRsq.pdf
> >> >
> >> > Merlo, J., Yang, M., Chaix, B., Lynch, J., & Råstam, L. (2005). A
> brief
> >> > conceptual tutorial on multilevel analysis in social epidemiology:
> >> > investigating contextual phenomena in different groups of people.
> Journal
> >> of
> >> > Epidemiology and Community Health, 59(9), 729-736. doi:
> >> > 10.1136/jech.2004.023929
> >> >
> >> > Orelien, J. G., & Edwards, L. J. (2008). Fixed-effect variable
> selection
> >> in
> >> > linear mixed models using R2 statistics. Computational Statistics &
> Data
> >> > Analysis, 52(4), 1896-1907. doi: 10.1016/j.csda.2007.06.006
> >> >
> >> > Roberts, J. K., & Monaco, J. P. (2006, April). Effect size measures
> for
> >> the
> >> > two-level linear multilevel model.  Paper presented at the annual
> meeting
> >> of
> >> > the American Educational Research Association, San Francisco, CA.
> >> Retrieved
> >> > from http://www.hlm-online.com/papers/HLM_effect_size.pdf
> >> >
> >> > Snijders, T. A. B., & Bosker, R. J. (1994). Modeled variance in
> two-level
> >> > models. Sociological Methods & Research, 22(3), 342-363. doi:
> >> > 10.1177/0049124194022003004
> >> >
> >> > Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis.
> London,
> >> UK:
> >> > Sage.
> >> > Xu, R. (2003). Measuring explained variation in linear mixed effects
> >> models.
> >> > Statistics in Medicine, 22(22), 3527-3541. doi: 10.1002/sim.1572
> >> >
> >> >
> >> >
> >> > Steven J. Pierce
> >> > Associate Director
> >> > Center for Statistical Training & Consulting (CSTAT)
> >> > Michigan State University
> >> > 178 Giltner Hall
> >> > East Lansing, MI 48824
> >> > Web: http://www.cstat.msu.edu
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: Luisa Carvalheiro [mailto:lgcarvalheiro@gmail.com]
> >> > Sent: Wednesday, February 17, 2010 6:00 AM
> >> > To: r-sig-mixed-models@r-project.org
> >> > Subject: [R-sig-ME] R2 measure in mixed models?
> >> >
> >> > Dear mixed modelers,
> >> >
> >> > Is there any package for calculating R2 measures for mixed models in R
> >> (e.g.
> >> > using the measure proposed by Mittlb ock;  Waldh or 2000)?
> >> >
> >> > Luisa
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Luisa Carvalheiro, PhD
> >> Southern African Biodiversity Institute, Kirstenbosch Research Center,
> >> Claremont
> >> & University of Pretoria
> >> Postal address - SAWC Pbag X3015 Hoedspruit 1380, South Africa
> >> telephone - +27 (0) 790250944
> >> Carvalheiro@sanbi.org
> >> lgcarvalheiro@gmail.com
> >>
> >> _______________________________________________
> >> R-sig-mixed-models@r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >>
> >
> >        [[alternative HTML version deleted]]
> >
> >
> > _______________________________________________
> > R-sig-mixed-models@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >
> >
>

	[[alternative HTML version deleted]]


