[R] ROCR - confidence interval for Sens and Spec

[Frank E Harrell Jr]

Rasanga Ruwanthi wrote:
> Dear List,
>  
> I am new to ROC analysis and the package ROCR. I want to compute the confidence intervals of sensitivity and specificity for a given cutoff value. I have used the following to calculate sensitivity and specificity:
>  
> data(ROCR.simple)
> pred <- prediction(ROCR.simple$predictions, ROCR.simple$labels)
>  
> se.sp <- function (cutoff, performance) { 
>     sens <- performance(pred,"sens") 
>     spec <- performance(pred,"spec") 
>     num.cutoff <- which.min(abs(sens at x.values[[1]] - cutoff))     
> return(list(Cutoff=sens at x.values[[1]][num.cutoff], Sensitivity=sens at y.values[[1]][num.cutoff], Specificity=spec at y.values [[1]][num.cutoff])) 
> } 
>  
> se.sp(cutoff=0.5, pred)
> 
>  
> I would be grateful if someone could let me know how to get 95% CIs for these sensitivity and specificity?
>  
> Thanks vermuchch in advance.
> Rasanga

The confidence interval will be meaningless unless you pre-specified a 
single cutoff without looking at the dataset.  You will probably have to 
use the bootstrap to penalize for the (severe) instability caused by 
examining multiple cutpoints.  Then you have to ask yourself why a 
cutoff is needed anyway.

Frank

>  
>  
>  
> 
> 
>       
> 	[[alternative HTML version deleted]]
> 
> 
> 
> ------------------------------------------------------------------------
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University