[R-sig-ME] Getting intraclass correlations from a binomial mixed model with logit link

Wed Aug 20 22:13:17 CEST 2014

Ulf,

More or less yes, but two things are worth emphasizing. First, although it is possible to construct a ratio like I / (S + I + E + pi^2/3) -- which you'll notice I did not put in my list -- I think the interpretation of this ratio would be dubious. It is tempting to say that it is the correlation for responses with the same Items, different Emotions, different Subjects. But note that you never actually observed any Items with more than one Emotion, so it's not totally clear that such a leap is warranted. Indeed, for this particular example (I guess these are items on a questionnaire of some kind), even the conceptual meaning of such a correlation is dubious, since it's not clear how the Items could be associated with different Emotions (although in other contexts this might not be hard to imagine). Second, all of this only works as long as there are only random intercepts in the model. As soon as there are random slopes as well, then everything becomes far more complicated and confusing and I basically would recommend not reporting ICCs in those cases.

Jake

Date: Wed, 20 Aug 2014 21:57:25 +0200
From: ukoether at uke.de
To: jake987722 at hotmail.com
Subject: Re: [R-sig-ME] Getting intraclass correlations from a binomial mixed model with logit link

    Dear Jake,

    thank you for the great answer! So, all in all, it is just as in the
    strictly nested random effects case and one has not to worry about
    mixing crossed and nested effects if I am understanding everything
    correctly. Nice!

    Regarding changing to a "manifest" variable instead of the latent
    variable view, that seems now clear to me. 

    To follow up on this for others who might be interested:

    Another paper addressing this issue seems to Rodriguez & Elo
    (2003), "Intra-class correlation in random-effects models for binary
    data", The Stata Journal, who also give functions in Stata to
    estimate the manifest ICC (thanks to Ben Pelzer for the hint). 

    One more paper that I found right after mailing to the list, is
    Nakagawa & Schielzeth (2010). "Repeatability for Gaussian and
    non-Gaussian

    data: a practical guide for biologists", Biological Reviews, who
    also wrote an R-package for this, "rptR"...

    Thanks again!

    Am 20.08.2014 um 21:30 schrieb Jake
      Westfall:

      Hi Ulf,

1. You can compute the following intraclass correlations. Let S (short for Subject) be the ID random intercept variance, I be the Item:Emotion random intercept variance, and E be the Emotion random intercept variance.

Correlation between responses with same Emotion, different Items, different IDs:
E / (S + I + E + pi^2/3)

Correlation between responses with same Emotion, same Items, different IDs:
(E + I) / (S + I + E + pi^2/3)

Correlation between responses with different Emotions, different Items, same IDs:
S / (S + I + E + pi^2/3)

Correlation between responses with same Emotions, different Items, same IDs:
(S + E) / (S + I + E + pi^2/3)

Correlation between responses with same Emotions, same Items, same IDs:
(S + I + E) / (S + I + E + pi^2/3)

2. The ICCs above are based on taking a latent variable view of the model. That is, we assume the responses arise from an underlying latent variable with a logistic distribution, and this logistic variable gets dichotomized around some threshold, so that we observe 0 below the threshold and 1 above the threshold. The ICCs above estimate various expected correlations in the value of this latent logistic variable. 

As hinted above, the intraclass correlation coefficient is, well, a bona fide correlation coefficient. So taking the inverse logit of a correlation doesn't really make sense.

The latent variable approach is nice because we don't have to specify a particular value of the predictor at which to assess the ICC. If you just want to talk about correlations involving the actually observed binary variable, with no latent variable baggage, you can do that, but you have to specify the expected value of Y that you're interested in (e.g., specify the values of all the predictors). That's because the variance of a binary variable depends on the mean, so accordingly the ICC is different for different expected Y values. In my opinion the notion of ICC loses its usefulness and intuitive appeal in this context. But if you want to compute it anyway, you can follow the simulation advice offered by Goldstein, Browne, & Rasbash, 2002, section 3.2.

http://www.bris.ac.uk/cmm/research/pvmm.pdf

Jake

        Date: Wed, 20 Aug 2014 19:30:30 +0200
From: ukoether at uke.de
To: r-sig-mixed-models at r-project.org
Subject: [R-sig-ME] Getting intraclass correlations from a binomial mixed model with logit link

Dear list members,

I am asking you for help on interpretating the random effects from a
binomial model (strictly 0-1 responses) with a logit link:

Random effects:
 Groups                     Name        Variance Std.Dev.
 ID                         (Intercept) 0.1475   0.3840  
 Item:Emotion               (Intercept) 2.7546   1.6597  
 Emotion                    (Intercept) 0.6822   0.8259  
Number of obs: 4788, groups:  ID, 114; Item:Emotion, 42; Emotion, 7

I would like to get an ICC for each random intercept, but there are some
conceptional problems here I cannot solve yet:

1.) ID and Item are completely crossed random effects, but Items are
nested within Emotion, so I do not know if I just can get an ICC for
each variance component via

sigma-ID^2 / (sigma-ID^2 + sigma-Item:Emotion^2 + sigma-Emotion^2 +
(pi^2/3))

with pi^2/3 as the "residual variance"-equivalent term for a binomial
logit model. The variance parts for the other random intercepts would be
calculated accordingly.

I read the Papers from Goldstein (2002) and Browne (2005) about
partitioning the variance but did not find any concrete hints about such
a model which consists of crossed and nested random effects.

2.) As a side question, I would like to know if one can get these ICCs
(if it is possible to get in the first place) on the probability scale
and not on the logit scale as presented in the model output? I assume
that just applying the inverse link function on the ICC would be no good
idea, but this is just a feeling... Does anyone know, why that is wrong?

Thanks for your help,

kind regards, Ulf

-- 
________________________________________

Dipl.-Psych. Ulf K�ther

PEPP-Team 
Klinik f�r Psychiatrie und Psychotherapie
Universit�tsklinikum Hamburg-Eppendorf
Martinistr. 52
20246 Hamburg

PEPP-Team:
Tel.: +49 (0) 40 7410 53248
pepp at uke.de

Pers�nlich:
Tel.: +49 (0) 40 7410 55851
Mobil: (9) 55851
ukoether at uke.de
________________________________________

--

DANKE F�R 125 JAHRE ENGAGEMENT UND VERTRAUEN.
www.uke.de/125
_____________________________________________________________________

Besuchen Sie uns auf: www.uke.de
_____________________________________________________________________

Universit�tsklinikum Hamburg-Eppendorf; K�rperschaft des �ffentlichen Rechts; Gerichtsstand: Hamburg
Vorstandsmitglieder: Prof. Dr. Christian Gerloff (Vertreter des Vorsitzenden), Prof. Dr. Dr. Uwe Koch-Gromus, Joachim Pr�l�, Rainer Schoppik
_____________________________________________________________________

SAVE PAPER - THINK BEFORE PRINTING

	[[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

	[[alternative HTML version deleted]]

      _______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

    -- 
________________________________________

Dipl.-Psych. Ulf Köther

PEPP-Team 
Klinik für Psychiatrie und Psychotherapie
Universitätsklinikum Hamburg-Eppendorf
Martinistr. 52
20246 Hamburg

PEPP-Team:
Tel.: +49 (0) 40 7410 53248
pepp at uke.de

Persönlich:
Tel.: +49 (0) 40 7410 55851
Mobil: (9) 55851
ukoether at uke.de
________________________________________

DANKE FÜR 125 JAHRE ENGAGEMENT UND VERTRAUEN.

www.uke.de/125
Besuchen Sie uns auf: www.uke.de

Universitätsklinikum Hamburg-Eppendorf; Körperschaft des öffentlichen Rechts; Gerichtsstand: Hamburg

Vorstandsmitglieder: Prof. Dr. Christian Gerloff (Vertreter des Vorsitzenden), Prof. Dr. Dr. Uwe Koch-Gromus, Joachim Prölß, Rainer Schoppik

SAVE PAPER - THINK BEFORE PRINTING 		 	   		  
	[[alternative HTML version deleted]]