[R] How to compare areas under ROC curves calculated with ROCR package

[Bernardo Rangel tura]

At 11:54 AM 3/23/2006, Frank Samuelson wrote:
>The seROC routine you included is an very good approximation to the
>standard error of the Mann-Whitney-Wilcoxon/Area under the ROC curve
>statistic.  It is derived from negative exponential models, but works
>very well in general (e.g. Hanley and McNeil, Diagnostic Radiology,
>1982, v. 143, p. 29).
>A more general estimator of the variance is given by Campbell,
>Douglas and Bailey, Proc. Computers in Cardiology, 1988, p.267)
>I've implemented that in R code included below.   It is not an unbiased
>estimator, but it is very close.
>
>The cROC function is probably not what you want, however.
>It assumes that the data from the two different area measures
>are independent.  You said your measures are "from the same dataset."
>Your different AUC measures will be highly correlated.
>There are a number of methods to deal with correlated ROC curves
>in existence.
>
>If you are interested in performing hypothesis testing on the difference
>in AUC of two parameters, I would suggest a permutation test.
>Permuting the ranks of the data between parameters is
>simple and works well.


Laurent, Jarek, Frank

My routines seROC and cROC was developed based in the paper

A method of comparing the areas under Receiver 
Operating Characteristic curves derived from the same cases.
Hanley JA & McNeil BJ, Radiology 1983 148 839-43

The seROC is calculation of standart error of ROC 
and cROC is calculation of statistical significance for two AUC in same sample.
I don´t documented this rotine but it assume that 
the data from the two different area measures are DEPENDENT.

A dependency of two measure is fix with 
incorporation of Pearson correlation among this measures.

I wait that this information helps yours to fix this problem.

Well... If you s think I can help yours just talk.



Bernardo Rangel Tura, MD, MSc
National Institute of Cardiology Laranjeiras
Rio de Janeiro Brazil