[R] Help with Krippendorff's Alpha

Jim Lemon drj|m|emon @end|ng |rom gm@||@com
Sat Jan 23 00:58:38 CET 2021

Hi Jean,
kripp.alpha expects a classifier by object (in your case, score)
matrix as the first argument. If I read your code correctly you are
getting the scores zigzagging across six columns when you want a 6 x
180 matrix. My guess is that you want:

Trainers<-matrix(c(Trainer_1,Trainer_2, Trainer_3, Trainer_5,
Trainer_6, Trainer_12),


On Sat, Jan 23, 2021 at 10:24 AM Jenn Russell <jenncami using hotmail.com> wrote:
> Hello!
> I am requesting help for a data set in which I have 6 evaluators (I've called them 'Trainers'), each of which took a questionnaire with 180 questions. The possible answers to the questions are ordinal numbers. I am looking to test inter-rater reliability of the questionnaire using Krippendorff's alpha. I have already run this in R, however when I get the output it seems the matrix is inverted. While it seems like it should be an easy problem to fix I don't see how I would be able to create my matrix the other way without typing out all 180 scores manually. For reference, I'm using the irr package to use kripp.alpha().
> > Trainers<-matrix(c(Trainer_1,Trainer_2, Trainer_3, Trainer_5, Trainer_6, Trainer_12), ncol=6)
> > kripp.alpha(Trainers, method=c("ordinal"))
> Krippendorff's alpha
> Subjects = 6
> Raters = 180
> alpha = -0.00149
> While I'm getting an output - shouldn't my raters be 6 and my subjects be 180?
> Appreciate the help!
> JR
>         [[alternative HTML version deleted]]
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

More information about the R-help mailing list