[R] 'singularity' between fixed effect and random factor in mixed model

[Douglas Bates]

On Thu, Jul 2, 2009 at 1:52 AM, Thomas Mang<thomas.mang at fiwi.at> wrote:
> Hi,

> I just came across the following issue regarding mixed effects models:
> In a longitudinal study individuals (variable ind) are observed for some
> response variable. One explanatory variable, f, entering the model as fixed
> effect, is a (2-level) factor. The expression of that factor is constant for
> each individual across time (say, the sex of the individual). ind enters the
> model as grouping variable for random effects. So in a simple form, the
> formula could look like:
> y ~ f + ... + (1|ind)
> [and in the simplest model, the ellipsis is simply nothing]

> To me, this seems not to be an unusual design at all.

> However, the indicator matrix consisting of f and ind - say if ind had
> entered the model as fixed effect - shows a singularity.

Yes.

> My question is now
> what will this 'singularity' cause in a mixed-effects model ? I admit, I
> have never fully understood how the fitting of mixed-effects models happen
> internally (whether REML or ML) [so I am not even sure if it can be called a
> 'singularity'].

You do not encounter a singularity in solving for the conditional
means of the random effects and the conditional estimates of the fixed
effects because there is a penalty assigned to the size of the random
effects vector.  This removes the ill-conditioning of the least
squares problem.  It is sometimes called "regularization" of the
estimation.

Should you wish to find out what does go on inside the lmer function
for REML or ML estimation of the parameters in a linear mixed model,
you can check out the slides from a short course that I just finished
at the University of Lausanne.  Go to

http://lme4.R-forge.R-project.org/slides

and click on the link "2009-07-01-Lausanne".  The display version of
the slides for the theory section, 6TheoryD.pdf, is the best
explanation I have yet been able to formulate for the theory.  The
important thing to note is that in the penalized linear least squares
problem the predictions for the "pseudo-observations" are affected by
the random effects but not by the fixed-effects.



> Specifically, does it make the fit numerically more unstable? Would the
> degree of this depend on other variables of the model? Is the issue of
> degrees of freedom - complicated enough anyway for mixed models - further
> inflated by that? Have statistical inferences regarding the fixed effect be
> treated more carefully? Is the general situation something that should be
> avoided ?
>
> many thanks in advance for any insights and cheers,
> Thomas
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>