[R] Pseudo R^2 for logit - really naive question

[Frank E Harrell Jr]

On Sun, 4 Aug 2002 11:23:34 -0500
Chris Lawrence <cnlawren at phy.olemiss.edu> wrote:

> On Aug 04, Paul M. Jacobson wrote:
> > I am using GLM to calculate logit models based on cross-sectional data.  I
> > am now down to the hard work of making the results intelligible to very
> > average readers.  Is there any way to calculate a psuedo analoque to the R^2
> > in standard linear regression for use as a purely descriptive statistic of
> > goodness of fit? Most of the readers of my report will be vaguely familiar
> > and more comfortable with R^2 than with any other regression diagnostics.
> 
> In fact, there are several "R^2-like" measures for logit and probit
> models (not surprisingly, called "pseudo-R^2").  An overview is in:
> 
> "Pseudo-R Measures for Some Common Limited Dependent Variable Models"
> http://citeseer.nj.nec.com/veall96pseudor.html

Chris - That's a really nice paper.  I hadn't realized that the index in Nagelkerke's 1991 Biometrika paper was first proposed by Cragg and Uhler in 1970.

> 
> The Aldrich-Nelson measure appears to be the most widely used.
> 
> You may also want to consider Herron's (1999) "Expected Percent
> Correctly Predicted" and related measures, described in Political
> Analysis 8(1): http://web.polmeth.ufl.edu/pa/herron.pdf; even
> traditional PCP/PRE measures tend to be quite informative (perhaps
> even more useful than Pseudo-R^2).

Percent correctly predicted has a host of problems, only some of which were pointed out in the above article.  This is an improper scoring rule (is not optimized when predicted probabilities are correct and its value can decrease when an important regressor is added to a model; see also http://hesweb1.med.virginia.edu/biostat/presentations/probclin.pdf).  The paper also did not reference the large literature on scoring rules for dichotomous outcomes (see e.g. work by Habbema and Hilden in the medical diagnostic literature and many papers on the Brier score and its decompositions).

Frank Harrell
> 
> 
> Chris
> -- 
> Chris Lawrence <cnlawren at olemiss.edu> - http://www.lordsutch.com/chris/
> 
> Instructor and Ph.D. Candidate, Political Science, Univ. of Mississippi
> 208 Deupree Hall - 662-915-5765
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._


-- 
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. Virginia School of Medicine  http://hesweb1.med.virginia.edu/biostat
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._