[R] Repeatability Analysis of Ordinal Data

[Sidoti, Salvatore]

Greetings, Colleagues:

I have several Likert-type ordinal data sets consisting of animal responses with repeated measures. I was able to implement a CLMM model easily enough with the package `ordinal`. However, the package does not support repeatability analyses.
 
Assuming that I subset my data according to treatment and/or sex, I am keen to try the `ordinalRR` package. According to the package documentation (https://cran.r-project.org/web/packages/ordinalRR/ordinalRR.pdf), performing `summary()` on the output from the function `ordinalRR()` returns the point estimates for each rater and for each pairwise combination of raters. However, is it possible to return an overall repeatability value and a 95% credible interval across all raters?

What follows is a stock procedure from the package reference document:

#-------------------------------------------------------------------------------
library(ordinalRR)

# load the dataset that comes with the package
data(followup)

# preprocess data to accommodate the package functions
followup.pre <- preprocess(followup)

# perform the analysis
followup.random <- ordinalRR(followup.pre)

summary(followup.random)

Call:
  ordinalRR(followup.pre)

Data: 30 parts, 3 operators, 2 repetitions with 4 ordinal categories.
Random-effects model MCMC chain: 1000 burn-in and 10000 retained.

Simple repeatability and model parameter estimates by rater:
  Rater j Repeatability  a_j d_{j,1} d_{j,2} d_{j,3}
1         0.900 12.0    -1.5    -0.1     0.6
2         0.900 10.9    -1.6    -0.3     0.5
3         0.933 12.7    -1.5    -0.2     0.5

Simple repeatability and reproducibility (R&R) point estimates for pairs of raters:
  Rater j Rater j' (R&R)_{j,j'}
1        2        0.808
1        3        0.900
2        3        0.850
#-------------------------------------------------------------------------------

Kind Regards,
Salvatore Sidoti
PhD Candidate
The Ohio State University
Columbus, Ohio USA