[R-sig-ME] [FORGED] Re: Using variance components of lmer for ICC computation in reliability study

[Doran, Harold]

That’s a helpful clarification, Rolf. However, with gaussian normal errors
in the linear model, we can’t *really* assume they would asymptote at 1 or
10. My suspicion is that these are likert-style ordered counts of some
form, although the OP should clarify. In which case, the 1 or 10 are
limits with censoring, as true values for some measured trait could exist
outside those boundaries (and I suspect the model is forming predicted
values outside of 1 or 10).

 

On 6/14/18, 6:33 PM, "Rolf Turner" <r.turner using auckland.ac.nz> wrote:

>
>On 15/06/18 05:35, Doran, Harold wrote:
>
>> Well no, you¹re specification is not right because your variable is not
>> continuous as you note. Continuous means it is a real number between
>> -Inf/Inf and you have boundaries between 1 and 10. So, you should not be
>> using a linear model assuming the outcome is continuous.
>
>I think that the foregoing is a bit misleading.  For a variable to be
>continuous it is not necessary for it to have a range from -infinity to
>infinity.
>
>The OP says that dv  "is a continuous variable (scale 1-10)".  It is not
>clear to me what this means.  The "obvious"/usual meaning or
>interpretation would be that dv can take (only) the (positive integer)
>values 1, 2, ..., 10.  If this is so, then a continuous model is not
>appropriate.  (It should be noted however that people in the social
>sciences do this sort of thing --- i.e. treat discrete variables as
>continuous --- all the time.)
>
>It is *possible* that dv can take values in the real interval [1,10], in
>which case it *is* continuous, and a "continuous model" is indeed
>appropriate.
>
>The OP should clarify what the situation actually is.
>
>cheers,
>
>Rolf Turner
>
>-- 
>Technical Editor ANZJS
>Department of Statistics
>University of Auckland
>Phone: +64-9-373-7599 ext. 88276
>
>> On 6/14/18, 11:16 AM, "Bernard Liew" <B.Liew using bham.ac.uk> wrote:
>> 
>>> Dear Community,
>>>
>>>
>>> I am doing a reliability study, using the methods of
>>> https://www.ncbi.nlm.nih.gov/pubmed/28505546. I have a question on the
>>> lmer formulation and the use of the variance components.
>>>
>>> Background: I have 20 subjects, 2 fixed raters, 2 testing sessions, and
>>> 10 trials per sessions. my dependent variable is a continuous variable
>>> (scale 1-10). Sessions are nested within each subject-assessor
>>> combination. I desire a ICC (3) formulation of inter-rater and
>>> inter-session reliability from the variance components.
>>>
>>> My lmer model is:
>>>
>>> lmer (dv ~ rater + (1|subj) + (1|subj:session), data = df)
>>>
>>> Question:
>>>
>>>   1.  is the model formulation right? and is my interpretation of the
>>> variance components for ICC below right?
>>>   2.  inter-rater ICC = var (subj) / (var(subj) + var (residual)) # I
>>> read that the variation of raters will be lumped with the residual
>>>   3.  inter-session ICC =( var (subj) + var (residual)) /( var (subj) +
>>> var (subj:session) + var (residual))
>>> some simulated data:
>>> df = expand.grid(subj = c(1:20), rater = c(1:2), session = c(1:2),
>>>trial
>>> = c(1:10))
>>> df$vas = rnorm (nrow (df_sim), mean = 3, sd = 1.5)
>>>
>>> I appreciate the kind response.
>
>