[R] What is a "good fit" Brier score and Harrel's C Index

[David Winsemius]

On Sep 28, 2013, at 8:14 AM, E Joffe wrote:

>
> Hi all,
>
> I am evaluating survival models using Brier score ("peperr") and  
> Harrel's
> C-Index ("Hmisc").

It's spelled 'Harrell'.

> I am wondering:
>
> 1. What would be considered a "good fit" according to these scores  
> (like the
> heuristic levels we have for R square in linear regressions) ?
>
> 2. Are there any papers to cite on the matter (I couldn't find any) ?

Frank Harrell's excellent text "Regression Modeling Strategies" has an  
extensive discussion of "goodness of fit" and the principles of model  
comparison. It's both too involved as well as off-topic for Rhelp. The  
other text to consult is Steyerberg's "Clinical Prediction Models".

>
> 3. Is there any paper to cite that discusses the limitation of using
> traditional reporting for model fit in survival analysis as opposed  
> to these
> measures  ?

I predict that the RMS bibliography would be an excellent place to  
start your search.

Despite getting his name attached to what he calls the 'c-index', I  
don't think one could call Frank Harrell a proponent of that measure  
or any of the "competitors". It's really just a dressed up/transformed  
AUC. The message I have taken from reading his book and listening to  
presentations is that one should apply biologic tests of sensibility  
as well as careful investigation of the functional relationships  
between candidate predictors and the outcomes of interest. He speaks  
very disparagingly about automatic procedures.

-- 

David Winsemius, MD
Alameda, CA, USA