[R-sig-ME] CLMM: Calculate ICC & Assessing Model Fit

[Juho Kristian Ruohonen]

And does the number of observations per animal vary? And if so, how many
observations are there per animal? If that number is sufficiently large, a
standard GLM with a fixed effect for animal ID might be enough.

As for the LRT statistic for the random effect, no it is not an effect
size. AFAIK, "effect size" is rarely talked about with random effects. What
I would view as the closest thing to the "effect size" of a random effect
is simply the random-effect SD, which reflects how much an "average"
individual/animal/cluster is estimated to deviate from the overall mean.
The ICC depends on this quantity, so the ICC could perhaps also be viewed
as a kind of "random effect size". But I couldn't tell you how the ICC is
calculated because that depends on the specifics of the cauchit link
function, with which I am not familiar. There are some link functions (such
as the probit) for which it is indeed as simple as SD^2 / (SD^2 + 1).

Best,

J



ke 19. elok. 2020 klo 17.09 Sidoti, Salvatore A. (
sidoti.23 using buckeyemail.osu.edu) kirjoitti:

> The example you see on Stack Exchange employs a toy data set which mimics
> the structure of my real data, but not the distributions. It turns out that
> the statistical contribution of each animal in my study is quite profound,
> so a mixed model is essential in my case.
>
> Salvatore Sidoti
> PhD Candidate
> Department of Evolution, Ecology & Organismal Biology
> The Ohio State University
> Columbus, OH USA
>
> ------------------------------
> *From:* Juho Kristian Ruohonen <juho.kristian.ruohonen using gmail.com>
> *Sent:* Wednesday, August 19, 2020, 9:35 AM
> *To:* Sidoti, Salvatore A.
> *Cc:* r-sig-mixed-models using r-project.org
> *Subject:* Re: [R-sig-ME] CLMM: Calculate ICC & Assessing Model Fit
>
> Just voicing two thoughts as a non-statistician:
>
> 1. Why do you feel you need a mixed model in the first place? It looks to
> me like every Individual is contributing the same number of responses (i.e.
> 4). Thus, the individual-specific effects can be assumed to cancel each
> other out, and no random effect is needed.
> 2. Why complicate things with a non-canonical link function? It looks to
> me like a standard ordinal logistic regression model, with fixed effects
> only, would do the job for you. Guides and textbooks abound on how to
> assess the fit of such models.
>
> Best,
>
> J
>
> ke 19. elok. 2020 klo 14.44 Sidoti, Salvatore A. (
> sidoti.23 using buckeyemail.osu.edu) kirjoitti:
>
>> To begin with, I'm not a fan of cross-posting. However, I posted my
>> question on Stack Exchange more than two weeks ago, but I have yet to
>> receive a sufficient answer:
>>
>>
>> https://stats.stackexchange.com/questions/479600/data-with-ordinal-responses-calculate-icc-assessing-model-fit
>> <https://urldefense.com/v3/__https://stats.stackexchange.com/questions/479600/data-with-ordinal-responses-calculate-icc-assessing-model-fit__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6iZNEz0M$>
>>
>> Here's what I've learned since then (hopefully):
>>
>> 1) ICC of a CLMM:
>> Computed like this:
>> (variance of the random effect) / (variance of the random effect + 1) If
>> this is correct, I would love to see a reference/citation for it.
>>
>> 2) 95% Confidence Interval for the ICC from a CLMM Model To my current
>> understanding, a confidence interval for an ICC is only obtainable via
>> simulation. I've conducted simulations with GLMM model objects ('lme4'
>> package) and the bootMer() function. Unfortunately, bootMer() will not
>> accept a CLMM model ('ordinal' package).
>>
>> 3) Model Fit of a CLMM
>> Assuming that the model converges without incident, the model summary
>> includes a condition number of the Hessian ('cond.H'). This value should be
>> below 10^4 for a "good fit". This is straightforward enough. However, I am
>> not as sure about the value for 'max.grad', which needs to be "well below
>> 1". The question is, to what magnitude should max.grad < 1 for a decent
>> model fit? My reference is linked below (Christensen, 2019), but it does
>> not elaborate further on this point:
>>
>>
>> https://documentcloud.adobe.com/link/track?uri=urn:aaid:scds:US:b6a61fe2-b851-49ce-b8b1-cd760d290636
>> <https://urldefense.com/v3/__https://documentcloud.adobe.com/link/track?uri=urn:aaid:scds:US:b6a61fe2-b851-49ce-b8b1-cd760d290636__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR65za2Jag$>
>>
>> 3) Effect Size of a CLMM
>> The random variable's effect is determined by a comparison between the
>> full model to a model with only the fixed effects via the anova() function.
>> I found this information on the 'rcompanion' package website:
>>
>> https://rcompanion.org/handbook/G_12.html
>> <https://urldefense.com/v3/__https://rcompanion.org/handbook/G_12.html__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6uPeTzGc$>
>>
>> The output of this particular anova() will include a value named
>> 'LR.stat', the likelihood ratio statistic. The LR.stat is twice the
>> difference of each log-likelihood (absolute value) of the respective
>> models. Is LR.stat the mixed-model version of an "effect size"? If so, how
>> does one determine if the effect is small, large, in-between, etc?
>>
>> Cheers,
>> Sal
>>
>> Salvatore A. Sidoti
>> PhD Candidate
>> Behavioral Ecology
>> The Ohio State University
>>
>> _______________________________________________
>> R-sig-mixed-models using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> <https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models__;!!KGKeukY!nOrbsi1ZjRJ1NgSNVsN8cyWBHD9f41iFHEvxXTqax6B_qnbhSquQQbwMHrEvhqWEvwR6KhTBL3M$>
>>
>
>

	[[alternative HTML version deleted]]