[R] Best performance measure?
noah at smartmediacorp.com
Wed Aug 19 19:23:38 CEST 2009
I working on a model to predict probabilities.
I don't really care about binary prediction accuracy.
I do really care about the accuracy of my probability predictions.
Frank was nice enough to point me to the val.prob function from the
Design library. It looks very promising for my needs.
I've put together some tests and run the val.prob analysis. It produces
some very informative graphs along with a bunch of performance measures.
Unfortunately, I'm not sure which measure, if any, is the "best" one.
I'm comparing hundreds of different models/parameter combinations/etc.
So Ideally I'd like a single value or two as the "performance measure"
for each one. That way I can pick the "best" model from all my
As mentioned above, I'm mainly interested in the accuracy of my
Does anyone have an opinion about which measure I should look at??
(I see Dxy, C, R2, D, U, Briar, Emax, Eavg, etc.)
More information about the R-help