Thats great thanks,

I suppose it is hard to move away from a more "traditional" measure of performance such a percentage correct, at least for the relatively amateur statisticians among us who have been graded on such a system.

The difficulty comes in reporting the effectiveness of the model to my peers. I have a c-value of 0.71 and a Bieber score of 0.199. So when it comes to predicting the response of newdata (using the estimated mean Y, which i am more comfortable understanding) 

i.e 

species 1  - mean = 2.12 therefore on the response scale this is 2
species 2  - mean = 2.98 therefore on the response scale this is 3
etc

(on a side note, using this approach no species had a mean of 6, which is the last ordinal category?)

A common question is how confident are you that the species has that response. 

What you are saying is using bootsrapping, and quoting "more proper scoring rules" such as c-values and Bieber score  explains the confidence sufficiently?

Chris




On 22 Sep 2010, at 12:36, Frank Harrell wrote:


% correct is an improper scoring rule and a discontinuous one to boot.  So it
will not always agree with more proper scoring rules.

When you have a more difficult task, e.g., discriminating more categories,
indexes such as the generalized c-index that utilize all the categories will
recognize the difficulty of the task and give a lower value.  No cause for
alarm.

Frank

-----
Frank Harrell
Department of Biostatistics, Vanderbilt University
-- 
View this message in context: http://r.789695.n4.nabble.com/predict-lrm-Design-package-tp2546894p2550118.html
Sent from the R help mailing list archive at Nabble.com.

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


	[[alternative HTML version deleted]]

