
Hi Stefan,

 

Thanks so much. This will help me i am sure. These past 2 days i was away on a trip so please excuse my delayed answer.

 

Monica
 
> From: stefan.evert@uos.de
> To: pisicandru@hotmail.com
> Subject: Re: [R] statistical significance of accuracy increase in classification
> Date: Thu, 26 Feb 2009 15:18:32 +0100
> CC: r-help@r-project.org
> 
> 
> On 26 Feb 2009, at 14:14, Max Kuhn wrote:
> 
> >> Do you know about any good reference that discusses kappa for 
> >> classification and maybe CI for kappa???
> 
> You might also want to take a look at this survey article on kappa and 
> its alternatives:
> 
> Artstein, Ron and Poesio, Massimo (2008). Survey article: Inter-coder 
> agreement for computational linguistics. Computational Linguistics, 
> 34(4), 555–596.
> 
> which you can download from
> 
> http://www.aclweb.org/anthology-new/J/J08/
> 
> Alternatives to the standard Fleiss-Cohen asymptotic confidence 
> intervals in the elementary 2x2 case are discussed in
> 
> Lee, J.J., Tu, Z. N.:"A Better Confidence for Kappa on Measuring 
> Agreement Between Two Raters with Binary Outcomes" Journal of 
> Computational and Graphical Statistics, 3:301-321, 1994.
> 
> which is available from JSTOR:
> 
> http://www.jstor.org/stable/1390914
> 
> An S implementation of their approximations can be downloaded here:
> 
> http://lib.stat.cmu.edu/S/kappa
> 
> I've started to evaluate the accuracy of these approximations with 
> simulation experiments some time ago, but haven't found the time to 
> follow up on it.
> 
> Hope this helps,
> Stefan
> 
> 

_________________________________________________________________


50F681DAD532637!7540.entry?ocid=TXT_TAGLM_WL_t2_ugc_post_022009
	[[alternative HTML version deleted]]


