[R-sig-ME] [FORGED] Re: Using variance components of lmer for ICC computation in reliability study

[Rolf Turner]

On 15/06/18 05:35, Doran, Harold wrote:

> Well no, you¹re specification is not right because your variable is not
> continuous as you note. Continuous means it is a real number between
> -Inf/Inf and you have boundaries between 1 and 10. So, you should not be
> using a linear model assuming the outcome is continuous.

I think that the foregoing is a bit misleading.  For a variable to be 
continuous it is not necessary for it to have a range from -infinity to 
infinity.

The OP says that dv  "is a continuous variable (scale 1-10)".  It is not 
clear to me what this means.  The "obvious"/usual meaning or 
interpretation would be that dv can take (only) the (positive integer) 
values 1, 2, ..., 10.  If this is so, then a continuous model is not 
appropriate.  (It should be noted however that people in the social 
sciences do this sort of thing --- i.e. treat discrete variables as 
continuous --- all the time.)

It is *possible* that dv can take values in the real interval [1,10], in
which case it *is* continuous, and a "continuous model" is indeed 
appropriate.

The OP should clarify what the situation actually is.

cheers,

Rolf Turner

-- 
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

> On 6/14/18, 11:16 AM, "Bernard Liew" <B.Liew using bham.ac.uk> wrote:
> 
>> Dear Community,
>>
>>
>> I am doing a reliability study, using the methods of
>> https://www.ncbi.nlm.nih.gov/pubmed/28505546. I have a question on the
>> lmer formulation and the use of the variance components.
>>
>> Background: I have 20 subjects, 2 fixed raters, 2 testing sessions, and
>> 10 trials per sessions. my dependent variable is a continuous variable
>> (scale 1-10). Sessions are nested within each subject-assessor
>> combination. I desire a ICC (3) formulation of inter-rater and
>> inter-session reliability from the variance components.
>>
>> My lmer model is:
>>
>> lmer (dv ~ rater + (1|subj) + (1|subj:session), data = df)
>>
>> Question:
>>
>>   1.  is the model formulation right? and is my interpretation of the
>> variance components for ICC below right?
>>   2.  inter-rater ICC = var (subj) / (var(subj) + var (residual)) # I
>> read that the variation of raters will be lumped with the residual
>>   3.  inter-session ICC =( var (subj) + var (residual)) /( var (subj) +
>> var (subj:session) + var (residual))
>> some simulated data:
>> df = expand.grid(subj = c(1:20), rater = c(1:2), session = c(1:2), trial
>> = c(1:10))
>> df$vas = rnorm (nrow (df_sim), mean = 3, sd = 1.5)
>>
>> I appreciate the kind response.