[R] Calculating Sensitivity, Specificity, and Agreement from Logistics Regression Model

[Frank E Harrell Jr]

Felix Schönbrodt wrote:
>> *Von: *Frank E Harrell Jr <f.harrell at vanderbilt.edu 
>> <mailto:f.harrell at vanderbilt.edu>>
>> *Datum: *18. Dezember 2008 14:49:53 MEZ
>> *An: *Meir Preiszler <pmeir at Itamar-Medical.com 
>> <mailto:pmeir at Itamar-Medical.com>>
>> *Kopie: *r-help at r-project.org <mailto:r-help at r-project.org>
>> *Betreff: **Re: [R] Calculating Sensitivity, Specificity, and 
>> Agreement from Logistics Regression Model*
>>
>>
>> Meir Preiszler wrote:
>>> Hi,
>>> Assume I have a variable Y having two discrete values and two 
>>> predictor variables x1 and x2.
>>> I then do a logistic regression model fit as:
>>> fit<-glm(Y~x1+x2,family=binomial). Are there functions in R than 
>>> calculate the
>>> Sensitivity, Specificity , and Agreement of the model "fit"?
>>> Thanks
>>> Meir
>>
>> Beware as those 3 measures are discontinuous functions of x1 and x2, 
>> requiring completely arbitrary dichtomizations, and are improper 
>> scoring rules in the statistical sense.
> 
> Hi Frank, maybe you should take a look at the ROCR package. I use it a 
> lot (as well with logistic regression), it can plot and calculate many 
> classification relevant indices.
> 
> Felix

Felix,

I don't know if ROCR deals with

   author =               {Pencina, Michael J. and {D'Agostino Sr}, 
Ralph B. and {D'Agostino Jr}, Ralph B. and Vasan, Ramachandran S.},
   title =                {Evaluating the added predictive ability of a 
new marker: {From} area under the {ROC} curve to reclassification and 
beyond},
   journal =      Stat in Med,
   year =                 2008,
   volume =               27,
   pages =        {157-172},
   annote =               {discrimination;model 
performance;AUC;C-index;risk prediction;biomarker;small differences in 
ROC area can still be very meaningful;example of insignificant test for 
difference in ROC areas with very significant results from new 
method;Yates' discrimination slope;reclassification table;limiting 
version of this based on whether and amount by which probabilities rise 
for events and lower for non-events when compare new model to 
old;comparing two models}
}

Pencina et al's methods are implemented in an upcoming new release of 
the Hmisc package.  For comparing two probability models, Pencina et 
al's approach is much more powerful than using ordinary sensitivity, 
specificity, and ROC area.

Frank

-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University