[R] ROC curve

[Claudia Beleites]

Dear Changbin,

> I want to know how to select the optimal decision threshold from the ROC
> curve? 
Depends on what optimal means. I think there are a bunch of different criteria used:

- point closest to the ideal model
- point furthest from the "guessing" model
- these criteria may include costs, i.e. a FP/FN ratio != 1
- ...

More practical:
If you use ROCR: the help of the performance class explains the slots in the 
object. You find there the data of the curve, incl. the thresholds.

> At what threshold will give the highest accuracy?
to know that, optmize the accuracy as function of the threshold.

Remember: finding the optimal threshold from a ROC curve is a data-driven 
optimization. You need to validate the resulting model with independent test 
data afterwards.



-- 
Claudia Beleites
Dipartimento dei Materiali e delle Risorse Naturali
Università degli Studi di Trieste
Via Alfonso Valerio 6/a
I-34127 Trieste

phone: +39 0 40 5 58-37 68
email: cbeleites at units.it