[R-sig-ME] [FORGED] Re: Using variance components of lmer for ICC computation in reliability study

Fri Jun 15 10:57:01 CEST 2018

It seems to me you actually have censored data of three types, left, right and within the intervals. You might find it helpful to review this paper and see how models like yours can be estimated.

https://cran.r-project.org/web/packages/censReg/vignettes/censReg.pdf

From: Bernard Liew <B.Liew using bham.ac.uk<mailto:B.Liew using bham.ac.uk>>
Date: Friday, June 15, 2018 at 1:20 AM
To: Ben Bolker <bbolker using gmail.com<mailto:bbolker using gmail.com>>, AIR <hdoran using air.org<mailto:hdoran using air.org>>
Cc: Rolf Turner <r.turner using auckland.ac.nz<mailto:r.turner using auckland.ac.nz>>, "r-sig-mixed-models using r-project.org<mailto:r-sig-mixed-models using r-project.org>" <r-sig-mixed-models using r-project.org<mailto:r-sig-mixed-models using r-project.org>>
Subject: RE: [R-sig-ME] [FORGED] Re: Using variance components of lmer for ICC computation in reliability study

Thanks all,

My original question appears to be now two (1) the distribution of my DV (hence what models to use; (2) specification of my lmer model to parse out variance components.

Topic (1): DV distribution

Yes, my measure is a sliding rule between 1-10 of subjective pain, so any number up to a single decimal is plausible. Is a linear model automatically excluded, or can (a) do a fitted/residual plot for checking; (b) log transform the dv if (a) shows evidence of  non-normality.

Going back to Rolf's point of social science, you are right. But realistically, many measures in biomechanics (which I am in), are analyzed using linear models, even though they are bounded. Example, a simple scalar height is bounded to a lower limit of zero, and an upper limit of what ever instrument is created. Joint angles are bounded physiologically too. So when are measures really -inf/inf?

Topic (2): lmer

Assuming my DV is appropriate for lmer, base on the experimental design used, I hope to receive some feedback on my fixed and random effects specification still 😊

Thanks again all, for the kind response

Bernard

-----Original Message-----
From: bbolker using gmail.com<mailto:bbolker using gmail.com> <bbolker using gmail.com<mailto:bbolker using gmail.com>>
Sent: Friday, June 15, 2018 2:28 AM
To: Doran, Harold <HDoran using air.org<mailto:HDoran using air.org>>
Cc: Rolf Turner <r.turner using auckland.ac.nz<mailto:r.turner using auckland.ac.nz>>; r-sig-mixed-models using r-project.org<mailto:r-sig-mixed-models using r-project.org>; Bernard Liew <B.Liew using bham.ac.uk<mailto:B.Liew using bham.ac.uk>>
Subject: Re: [R-sig-ME] [FORGED] Re: Using variance components of lmer for ICC computation in reliability study

More generally, the best way to fit this kind of model is to use an

*ordinal* model, which assumes the responses are in increasing sequence but does not assume the distance between levels (e.g. 1 vs 2,

2 vs 3 ...) is uniform.  However, I'm not sure how one would go about computing an ICC from ordinal data ... (the 'ordinal' package is the place to look for the model-fitting procedures). Googling it finds some stuff, but it seems that it doesn't necessarily apply to complex designs ...

https://stats.stackexchange.com/questions/3539/inter-rater-reliability-for-ordinal-or-interval-data

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3402032/

On Thu, Jun 14, 2018 at 6:58 PM, Doran, Harold <HDoran using air.org<mailto:HDoran using air.org>> wrote:

> That’s a helpful clarification, Rolf. However, with gaussian normal

> errors in the linear model, we can’t *really* assume they would

> asymptote at 1 or 10. My suspicion is that these are likert-style

> ordered counts of some form, although the OP should clarify. In which

> case, the 1 or 10 are limits with censoring, as true values for some

> measured trait could exist outside those boundaries (and I suspect the

> model is forming predicted values outside of 1 or 10).

>

>

>

> On 6/14/18, 6:33 PM, "Rolf Turner" <r.turner using auckland.ac.nz<mailto:r.turner using auckland.ac.nz>> wrote:

>

>>

>>On 15/06/18 05:35, Doran, Harold wrote:

>>

>>> Well no, you¹re specification is not right because your variable is

>>> not continuous as you note. Continuous means it is a real number

>>> between -Inf/Inf and you have boundaries between 1 and 10. So, you

>>> should not be using a linear model assuming the outcome is continuous.

>>

>>I think that the foregoing is a bit misleading.  For a variable to be

>>continuous it is not necessary for it to have a range from -infinity

>>to infinity.

>>

>>The OP says that dv  "is a continuous variable (scale 1-10)".  It is

>>not clear to me what this means.  The "obvious"/usual meaning or

>>interpretation would be that dv can take (only) the (positive integer)

>>values 1, 2, ..., 10.  If this is so, then a continuous model is not

>>appropriate.  (It should be noted however that people in the social

>>sciences do this sort of thing --- i.e. treat discrete variables as

>>continuous --- all the time.)

>>

>>It is *possible* that dv can take values in the real interval [1,10],

>>in which case it *is* continuous, and a "continuous model" is indeed

>>appropriate.

>>

>>The OP should clarify what the situation actually is.

>>

>>cheers,

>>

>>Rolf Turner

>>

>>--

>>Technical Editor ANZJS

>>Department of Statistics

>>University of Auckland

>>Phone: +64-9-373-7599 ext. 88276

>>

>>> On 6/14/18, 11:16 AM, "Bernard Liew" <B.Liew using bham.ac.uk<mailto:B.Liew using bham.ac.uk>> wrote:

>>>

>>>> Dear Community,

>>>>

>>>>

>>>> I am doing a reliability study, using the methods of

>>>> https://www.ncbi.nlm.nih.gov/pubmed/28505546. I have a question on

>>>> the lmer formulation and the use of the variance components.

>>>>

>>>> Background: I have 20 subjects, 2 fixed raters, 2 testing sessions,

>>>> and

>>>> 10 trials per sessions. my dependent variable is a continuous

>>>> variable (scale 1-10). Sessions are nested within each

>>>> subject-assessor combination. I desire a ICC (3) formulation of

>>>> inter-rater and inter-session reliability from the variance components.

>>>>

>>>> My lmer model is:

>>>>

>>>> lmer (dv ~ rater + (1|subj) + (1|subj:session), data = df)

>>>>

>>>> Question:

>>>>

>>>>   1.  is the model formulation right? and is my interpretation of

>>>>the  variance components for ICC below right?

>>>>   2.  inter-rater ICC = var (subj) / (var(subj) + var (residual)) #

>>>>I  read that the variation of raters will be lumped with the residual

>>>>   3.  inter-session ICC =( var (subj) + var (residual)) /( var

>>>>(subj) +  var (subj:session) + var (residual))  some simulated data:

>>>> df = expand.grid(subj = c(1:20), rater = c(1:2), session = c(1:2),

>>>>trial  = c(1:10))  df$vas = rnorm (nrow (df_sim), mean = 3, sd =

>>>>1.5)

>>>>

>>>> I appreciate the kind response.

>>

>>

>

> _______________________________________________

> R-sig-mixed-models using r-project.org<mailto:R-sig-mixed-models using r-project.org> mailing list

> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

	[[alternative HTML version deleted]]