
On 09.06.2009, at 00:09, David Duffy wrote:

> On Mon, 8 Jun 2009, Reinhold Kliegl wrote:
>
>> Conditional modes (generated from the model parameters and the  
>> data) are not independent observations. Therefore, only the second  
>> method is valid.
>>
>> Reinhold Kliegl
>>
>> On 08.06.2009, at 13:04, Jeroen Ooms wrote:
>>
>>> I have tried to use lme4 to analyze IRT like datasets, but now I am
>>> confused. I have a data set with intelligence items (i.e. score 0  
>>> or 1), for
>>> completely crossed subjects and items. Furthermore, the data  
>>> contains some
>>> personality scores on the subject level. Actually the data is more
>>> complicated than this, but let's keep it simple for now. My research
>>> question is whether a personality charcteristic, say extraversion,  
>>> is
>>> related to intelligence. My question is how I should incorporate the
>>> extraversion variable in the analysis.
>>> When I analyse this data using the Rasch model, I usually first  
>>> fit the
>>> model, then extract the 'latent trait scores', and relate these to  
>>> the
>>> extraversion scores. I could do the same with lmer:
>>> myModel <- lmer(y~1+(1|item)+(1|subject),data=mydata,  
>>> family=binomial);
>>> intelligence <- ranef(myModel)$subject[[1]];
>>> lm(intelligence~extraversion);
>>> However, in the context of multilevel analysis, it is also  
>>> possible to
>>> incorporate the extraversion variable directly into the model:
>>> myModel2 <- lmer(y~1+(1|item)+(1|subject)+extraversion,data=mydata,
>>> family=binomial);
>>> Conceptually both methods feel very similar, but they give different
>>> results. What is the most appropriate method? What are the  
>>> differences in
>>> interpretation?
>>
>
> Aside from Reinhold's comment, which is not a showstopper (you could  
> bootstrap etc), they are quite different models. In the first model,  
> the estimated IQ-extraversion correlation is disattenuated for  
> measurement error -- the equivalent SEMs are something like (I  
> think ;)):
>
>
> IQ <-----> E  v.   E
> | \                | \
> v  v               v  v
> i1 i2              i1 i2
>                   ^  ^
>                   | /
>                   IQ
>
> Most people would prefer something like the first model, and in fact  
> would
> estimate the correlation between IQ and E estimated (as if without  
> error) from two measurement models given by the scoring rules for  
> the instruments (these are essentially BLUPs).  Incorporating  
> measurement error for both measures is the truest way to do it.
>
> You could compare results using the sem and polycor packages to  
> those from your lmer model.
>
> my 2c, David Duffy.
>
Always interested to be corrected...

My comment was referring to the use of conditional modes (formerly  
known as BLUPs)--extracted with ranef()--not bootstrap or SEM. Not  
sure to what degree these alternatives correspond to the use of  
conditional modes.  As far as conditional modes are concerned, we  
(Kliegl, Masson, & Richter)  have a paper in press (available at my  
publication page), in which we write the following:

"Conditional means have much appeal because they take into account  
differences in the reliability of the grouped data. As a stern warning  
against treating them as independent observations, however, we  
document with simulations that it is indeed the LMM estimates, not the  
conditional means predicted from them, that we have to rely on for  
inference. To this end, we generated 100,000 sets of data for a simple  
LMM model including 30 “subjects” and a predictor with 10 levels,  
conforming to a known variance for intercept and slope across subjects  
and varying the true correlation between these parameters from -0.9 to  
+0.9 in 2,000 steps (i.e., each simulation used a different  
correlation). ...
... [C]onditional means underestimate variances and exaggerate  
covariances and correlations. The shrinkage of variance reflects the  
contribution of the likelihood in the computation of conditional  
means. Shrinkage correction for predictions leads to dampening of the  
variance components, but, as we have shown in this section, not of the  
associated covariance component. The shrinkage of variance prevents  
overfitting of unreliable data but, as a curious side effect, the  
“correlations” based on conditional means for individual subjects are  
larger in absolute value than the corresponding LMM estimates of the  
correlation.
        In summary, the simulation shows that model estimates (top)  
are fine, even when estimates are based on only 50% of the data, but  
conditional means (bottom), because of their dependency on estimated  
population values, clearly overestimate the magnitude of the  
correlation estimate. Thus, interpretations of correlations must not  
be based on the “correlations” computed from conditional means, but on  
LMM estimates of correlations."

I am sorry, for this excessive self-citation, but I would rather find  
out asap whether I should dampen this "prescription", ideally before  
the paper appears.  It is certainly not in my interest to mislead the  
community.

Best
Reinhold

PS: I also think that including covariates in the model from the  
beginning is conceptually more appealing than the two-step process.  
Was it not one of the major advances of SEMs that coefficients of  
measurement models for latent variables and structural model  
coefficients could be estimated simultaneously?



	[[alternative HTML version deleted]]


