[R] ROC optimal threshold

[Tim Howard]

Jose - 

I've struggled a bit with the same question, said another way: "how do you find the value in a ROC curve that minimizes false positives while maximizing true positives"?

Here's something I've come up with. I'd be curious to hear from the list whether anyone thinks this code might get stuck in local minima, or if it does find the global minimum each time. (I think it's ok).

>From your ROC object you need to grab the sensitivity (=true positive rate) and specificity (= 1- false positive rate) and the cutoff levels.  Then find the value that minimizes abs(sensitivity-specificity), or  sqrt((1-sens)^2)+(1-spec)^2)) as follows:

absMin <- extract[which.min(abs(extract$sens-extract$spec)),];
sqrtMin <- extract[which.min(sqrt((1-extract$sens)^2+(1-extract$spec)^2)),];

In this example, 'extract' is a dataframe containing three columns: extract$sens = sensitivity values, extract$spec = specificity values, extract$votes = cutoff values. The command subsets the dataframe to a single row containing the desired cutoff and the sens and spec values that are associated with it.

Most of the time these two answers (abs or sqrt) are the same, sometimes they differ quite a bit. 

I do not see this application of ROC curves very often. A question for those much more knowledgeable than I.... is there a problem with using ROC curves in this manner?

Tim Howard




Date: Fri, 31 Mar 2006 11:58:14 +0200
From: "Anadon Herrera, Jose Daniel" <jdanadon at umh.es>
Subject: [R] ROC optimal threshold
To: "'r-help at stat.math.ethz.ch'" <r-help at stat.math.ethz.ch>
Message-ID:
	<79C6D1A4DD5E7B46B663C43C0021236556F66D at mailer-e071.umh.es>
Content-Type: text/plain;	charset=iso-8859-1

hello,

I am using the ROC package to evaluate predictive models
I have successfully plot the ROC curve, however

?is there anyway to obtain the value of operating point=optimal threshold
value (i.e. the nearest point of the curve to the top-left corner of the
axes)?

thank you very much,


jose daniel anadon
area de ecologia
universidad miguel hernandez

espa?a