[R-sig-ME] Using variance components of lmer for ICC computation in reliability study

[Bernard Liew]

Dear Community,


I am doing a reliability study, using the methods of https://www.ncbi.nlm.nih.gov/pubmed/28505546. I have a question on the lmer formulation and the use of the variance components.

Background: I have 20 subjects, 2 fixed raters, 2 testing sessions, and 10 trials per sessions. my dependent variable is a continuous variable (scale 1-10). Sessions are nested within each subject-assessor combination. I desire a ICC (3) formulation of inter-rater and inter-session reliability from the variance components.

My lmer model is:

lmer (dv ~ rater + (1|subj) + (1|subj:session), data = df)

Question:

  1.  is the model formulation right? and is my interpretation of the variance components for ICC below right?
  2.  inter-rater ICC = var (subj) / (var(subj) + var (residual)) # I read that the variation of raters will be lumped with the residual
  3.  inter-session ICC =( var (subj) + var (residual)) /( var (subj) + var (subj:session) + var (residual))
some simulated data:
df = expand.grid(subj = c(1:20), rater = c(1:2), session = c(1:2), trial = c(1:10))
df$vas = rnorm (nrow (df_sim), mean = 3, sd = 1.5)

I appreciate the kind response.

Kind regards,
Bernard



	[[alternative HTML version deleted]]