[R-sig-ME] CLMM: Calculate ICC & Assessing Model Fit

[Sidoti, Salvatore A.]

The example you see on Stack Exchange employs a toy data set which mimics the structure of my real data, but not the distributions. It turns out that the statistical contribution of each animal in my study is quite profound, so a mixed model is essential in my case.

Salvatore Sidoti
PhD Candidate
Department of Evolution, Ecology & Organismal Biology
The Ohio State University
Columbus, OH USA

________________________________
From: Juho Kristian Ruohonen <juho.kristian.ruohonen using gmail.com>
Sent: Wednesday, August 19, 2020, 9:35 AM
To: Sidoti, Salvatore A.
Cc: r-sig-mixed-models using r-project.org
Subject: Re: [R-sig-ME] CLMM: Calculate ICC & Assessing Model Fit

Just voicing two thoughts as a non-statistician:

1. Why do you feel you need a mixed model in the first place? It looks to me like every Individual is contributing the same number of responses (i.e. 4). Thus, the individual-specific effects can be assumed to cancel each other out, and no random effect is needed.
2. Why complicate things with a non-canonical link function? It looks to me like a standard ordinal logistic regression model, with fixed effects only, would do the job for you. Guides and textbooks abound on how to assess the fit of such models.

Best,

J

ke 19. elok. 2020 klo 14.44 Sidoti, Salvatore A. (sidoti.23 using buckeyemail.osu.edu<mailto:sidoti.23 using buckeyemail.osu.edu>) kirjoitti:
To begin with, I'm not a fan of cross-posting. However, I posted my question on Stack Exchange more than two weeks ago, but I have yet to receive a sufficient answer:

https://stats.stackexchange.com/questions/479600/data-with-ordinal-responses-calculate-icc-assessing-model-fit<https://urldefense.com/v3/__https://stats.stackexchange.com/questions/479600/data-with-ordinal-responses-calculate-icc-assessing-model-fit__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6iZNEz0M$>

Here's what I've learned since then (hopefully):

1) ICC of a CLMM:
Computed like this:
(variance of the random effect) / (variance of the random effect + 1) If this is correct, I would love to see a reference/citation for it.

2) 95% Confidence Interval for the ICC from a CLMM Model To my current understanding, a confidence interval for an ICC is only obtainable via simulation. I've conducted simulations with GLMM model objects ('lme4' package) and the bootMer() function. Unfortunately, bootMer() will not accept a CLMM model ('ordinal' package).

3) Model Fit of a CLMM
Assuming that the model converges without incident, the model summary includes a condition number of the Hessian ('cond.H'). This value should be below 10^4 for a "good fit". This is straightforward enough. However, I am not as sure about the value for 'max.grad', which needs to be "well below 1". The question is, to what magnitude should max.grad < 1 for a decent model fit? My reference is linked below (Christensen, 2019), but it does not elaborate further on this point:

https://documentcloud.adobe.com/link/track?uri=urn:aaid:scds:US:b6a61fe2-b851-49ce-b8b1-cd760d290636<https://urldefense.com/v3/__https://documentcloud.adobe.com/link/track?uri=urn:aaid:scds:US:b6a61fe2-b851-49ce-b8b1-cd760d290636__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR65za2Jag$>

3) Effect Size of a CLMM
The random variable's effect is determined by a comparison between the full model to a model with only the fixed effects via the anova() function. I found this information on the 'rcompanion' package website:

https://rcompanion.org/handbook/G_12.html<https://urldefense.com/v3/__https://rcompanion.org/handbook/G_12.html__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6uPeTzGc$>

The output of this particular anova() will include a value named 'LR.stat', the likelihood ratio statistic. The LR.stat is twice the difference of each log-likelihood (absolute value) of the respective models. Is LR.stat the mixed-model version of an "effect size"? If so, how does one determine if the effect is small, large, in-between, etc?

Cheers,
Sal

Salvatore A. Sidoti
PhD Candidate
Behavioral Ecology
The Ohio State University

_______________________________________________
R-sig-mixed-models using r-project.org<mailto:R-sig-mixed-models using r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models<https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6KhTBL3M$>


	[[alternative HTML version deleted]]