[R] Cohen's kappa, unequal score ranges
Julia Myatt
JPM601 at bham.ac.uk
Wed Dec 23 17:25:05 CET 2009
Hi, Sorry, it's ok I've figured it out using the as.matrix function!
Cheers,
Julia.
________________________________________
From: Scot W. McNary [smcnary at charm.net]
Sent: 23 December 2009 15:41
To: Julia Myatt
Cc: r-help at r-project.org
Subject: Re: [R] Cohen's kappa, unequal score ranges
Julia,
I was able to get your toy data to work, using the n * 2 data frame,
which is what ckappa takes as input.
> chk <- data.frame(matrix(c(
+ 3, 4,
+ 4, 3,
+ 2, 1,
+ 2, 1,
+ 5, 1,
+ 2, 4), byrow = TRUE, ncol = 2))
> chk
X1 X2
1 3 4
2 4 3
3 2 1
4 2 1
5 5 1
6 2 4
> sk <- ckappa(chk)
> sk
$table
1 2 3 4 5
1 0 0 0 0 0
2 2 0 0 1 0
3 0 0 0 1 0
4 0 0 1 0 0
5 1 0 0 0 0
$kappa
[1] -0.0909091
> str(sk)
List of 2
$ table: num [1:5, 1:5] 0 2 0 0 1 0 0 0 0 0 ...
..- attr(*, "dimnames")=List of 2
.. ..$ : chr [1:5] "1" "2" "3" "4" ...
.. ..$ : chr [1:5] "1" "2" "3" "4" ...
$ kappa: num -0.091
Hope this helps,
Scot
Julia Myatt wrote:
> Hi Jim,
>
> Thanks for your help, I tried that function which did result in an output. The only issue now is my summary looks like this:
>
> Length Class Mode
> table 16 -none- numeric
> kappa 1 -none- numeric
>
> Not what I was expecting, this was the case when my data was in the following format:
>
> ob1 ob2
> 1 3 4
> 2 4 3
> 3 2 1
> 4 2 1
> 5 5 1
> 6 2 4
>
> I then tried entering my data as a matrix like the one below (all this data is made up until I get it to work!):
>
> p1 p2 p3 p4 p5
> p1 3 1 0 4 2
> p2 3 5 0 7 1
> p3 2 3 4 6 3
> p4 4 1 0 4 0
> p5 8 2 0 2 5
>
> and this resulted in the same type of output:
>
> Length Class Mode
> table 36 -none- numeric
> kappa 1 -none- numeric
>
> If that is the kappa output (1) something isn't right as I purposely made the data without good interrater reliability! I was wondering if you have anymore tips as to what I am doing wrong? I'm afraid I'm very new to R so don't even know the basics!!
>
> Thanks for all your help,
>
> Julia.
>
>
> ________________________________________
> From: Jim Lemon [jim at bitwrit.com.au]
> Sent: 23 December 2009 02:15
> To: Julia Myatt
> Cc: r-help at r-project.org
> Subject: Re: [R] Cohen's kappa, unequal score ranges
>
> On 12/23/2009 01:58 AM, Julia Myatt wrote:
>
>> Hi,
>>
>> I am having problems getting cohen's kappa to work. I have been using the function:
>>
>>
>>
>>> <-ckappa(x,y)
>>>
>>>
>> from the psy package.
>>
>> I am trying to test for inter-observer reliability, I have 2 observers and 26 categories, however, the two observers might not necessarily have the same range of categories (I have unequal score ranges). However, I thought R could cope with this.
>>
>> Each time I try the function above I get the following output:
>>
>>
>>
>>> Error in ckappa(ob1, ob2) : unused argument(s) (ob2)
>>>
>>>
>> I got this even when I altered the data to have equal score ranges, therefore I am not sure what this means or what I should do!!
>>
>> Any help in this area would be much appreciated, or anything about the best way to deal with inter-observer reliability (my data is all categorical),
>>
>>
>>
> Hi Julia,
> The ckappa function only takes one argument, a rater by object matrix or
> dataframe of scores. Try:
>
> ckappa(cbind(ob1,ob2))
>
> Jim
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
--
Scot McNary
smcnary at charm dot net
More information about the R-help
mailing list