[R-sig-ME] Error in mer_finalize(ans) : Downdated X'X is not positive definite, 9.

[Douglas Bates]

On Fri, Mar 6, 2009 at 3:44 AM, Jukka Corander <jukka.corander at abo.fi> wrote:
> Dear all,
>
> I tried to fit several models with lmer, some of which yield the following
> error message: Error in mer_finalize(ans) : Downdated X'X is not positive
> definite, 9.
> The experimental data are as follows:
> 3 factors: Sexual_orientat (2 values), X_type (4 values), Y_type (4 values)
> response lgRT is assumed Gaussian,
> each participant has been measured in multiple trials (~190) and there are
> 23 participants in the study.
> Design is not balanced, because the cases where X_type and Y_type have the
> same value (Y_type=1&X_type=1 etc) are excluded.
> Any model without an interaction term Y_type*X_type can be fitted without
> problems using lmer, but all models with that term (including 3-way
> interaction) yield the error message.
> For instance,
> FittedModel4 <- lmer(lgRT ~ Y_type*Sexual_orientat + X_type*Sexual_orientat
> + (1|Subject),data=arraydata)
> works fine, but
> FittedModel5 <- lmer(lgRT ~ Y_type*X_type*Sexual_orientat +
> (1|Subject),data=arraydata)
> yields the error. If the repeated nature of the measurements is ignored and
> a 3-way interaction model is fitted using ordinary anova, it goes through,
> but, of course, estimates cannot be obtained for all interaction terms
> because they are lacking in the design.

There is special code in the lm.fit function (that actual "workhorse"
that is called by lm) to handle a rank-deficient model matrix. A
specially modified version of the Linpack (not Lapack) dqrdc
subroutine is used to detect and deal with rank deficiency.  There is
no similar code available for the Cholesky decomposition that is used
in lmer so it just declares that the X'X matrix is rank deficient
after downdating.  (In this case it was rank deficient even before
downdating.)

>  Is the error due to the inbalanced
> design, or is there something else I've ignored/done wrong? Any insights are
> highly appreciated!