[R] Help with sample size calculation for inter-rater reliability study

Edison Iglesias de Oliveira eiovidal at hotmail.com
Sat Nov 8 19:34:25 CET 2014

```Dear R masters,

I am attempting to calculate the sample size for an inter-rater reliability
study using the N2.cohen.kappa function of the irr package.

It is a study of 2 raters for a single item with three possible ordinal
outcomes. The expected marginal probabilities for those outcomes are 0.2,
0.6 and 0.2.

The null hypothesis is that kappa < 0.8
The alternative hypotheis is kappa >=0.8

The following example comes from the irr package manual

require(lpSolve)
# Testing H0: kappa = 0.4 vs. HA: kappa > 0.4 (=0.6) given that
# Marginal Probabilities by two raters are (0.2, 0.25, 0.55).
#
# one sided test with 80% power:
N2.cohen.kappa(c(0.2, 0.25, 0.55), k1=0.6, k0=0.4)
# one sided test with 90% power:
N2.cohen.kappa(c(0.2, 0.25, 0.55), k1=0.6, k0=0.4, power=0.9)
# Marginal Probabilities by two raters are (0.2, 0.05, 0.2, 0.05, 0.2,
0.3)
# Testing H0: kappa = 0.1 vs. HA: kappa > 0.1 (=0.5) given that
#
# one sided test with 80% power:
N2.cohen.kappa(c(0.2, 0.05, 0.2, 0.05, 0.2, 0.3), k1=0.5, k0=0.1)

In my case I would be testing H0: kappa < 0.8 vs HA: kappa >= 08 given
Marginal probabilities by two raters as (0.2, 0.6, 0.2)

However the following argument will not work

N2.cohen.kappa (mrg=c(0.2,0.6,0.2), k1>=0.8, k0<0.8, alpha=0.05, power= 0.8,
twosided=FALSE)

I have also tried the kappaSize package without success

Power3Cats(kappa0<0.8, kappa1>=0.8, props=c(0.2,0.6,0.2), raters=2,
alpha=0.05, power=0.80)

Can anyone offer me some guidance?

Best regards,

Edison
[[alternative HTML version deleted]]

```