[R] Credit Scoring Model - SPEC (specificity) and SENS (sensitivity)

[Frank E Harrell Jr]

Maithili Shiva wrote:
> Dear R helpers,
> 
> Hi I am working on credit scoring model using logistic regression. I have main sample of 42500 clentes and based on their status as regards to defaulted / non - defaulted, I have genereted the probability of default.
> 
> I have a hold out sample of 5000 clients. I have calculated (1) No of correctly classified goods Gg, (2) No of correcly classified Bads Bg and also (3) number of wrongly classified bads (Gb) and (4) number of wrongly classified goods (Bg).
> 
> My prolem is how to interpret these results? What I have arrived at are the absolute figures. Using these I hav ecalculated Specificity (SPEC) and sensitivity (SENS) as
> 
> SPEC = Bb / (Bb + Gg)
> 
> and SENS = Gg / (Gg + Bg)
> 
> 
> With regards
> 
> Maithili

Sensitivity and specificity have no usefulness in your situation as they 
are in reverse time order (condition on the unknown and fail to 
condition on what was already known).  Your test sample is too small. 
You are not addressing absolute calibration through precise 
high-resolution methods.  My book Regression Modeling Strategies goes 
into this.

Frank
> 
-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University