[R] Performance measure for probabilistic predictions

[Noah Silverman]

Hello,

I'm using an SVM for predicting a model, but I'm most interested in the 
probability output.  This is easy enough to calculate.

My challenge is how to measure the relative performance of the SVM for 
different settings/parameters/etc.

An AUC curve comes to mind, but I'm NOT interested in predicting true vs 
false.  I am interested in finding the most accurate probability 
predictions possible.

I've seen some literature where the probability range is cut into 
segments and then the predicted probability is compared to the actual.  
This looks nice, but I need a more tangible numeric measure.  One 
thought was a measure of "probability accuracy" for each range, but how 
to calculate this.

Any thoughts?

-N