[R-sig-ME] Correlation of Fixed Effects

[Kingsford Jones]

On Mon, Feb 9, 2009 at 2:52 PM, Gorjanc Gregor
<Gregor.Gorjanc at bfro.uni-lj.si> wrote:
> Hi!
>
> The default print method outputs also Correlation of Fixed Effects.
> How is this computed and what does it actually represent?

Hi Gregor,

In a standard LM it's calculated Cov(\beta) = \sigma^{2}(X'IX)^{-1},
where X is the model design matrix.  In practice \sigma^2 is estimated
by the sum of squared residuals divided by the number of cols in X
minus its rank.

Although I'm guessing here, I assume the equation changes for an LMM
in that the estimate of \sigma^2 becomes a sum of estimated variance
components, and rather than an identity matrix, there may be any
positive definite matrix in between the X' and the X (e.g. if weights
or corr arguments are used in lme we get non-Identity error
covariance).  I tried to confirm this 'guess' by looking at the code
for the vcov method for mer objects, but my S4 skills are too limited
to know how to find it ---  anyone?  (and on a side note -- the fact
that this can (usually) be easily done is yet another reason why I
would be very happy to see Doug's future work to remain in R ;-))

As far as what it represents, as you'd guess the sqrt of the diagonals
are the SEs for the estimated coefficients and the off-diagonals are
the estimated covariances between those estimates.  I suppose another
answer is that the off-diagonals provide indication of the amount of
collinearity in X.

> I have two models
> that essentially give me the same message, but in one model the correlations
> between covariates are really high 0.9 and higher, while in other model use of
> poly(), reduced correlations a lot! Should I care?

Not surprisingly, when X has columns that are higher-order terms of
another column collinearity occurs and the correlations between
coefficients are high.  'poly' produces orthogonal polynomials, so
covariances of the resulting coefficients should be essentially zero.
The nice thing about that is that terms can be added/removed from the
model without affecting the remaining estimates.  On the other hand,
when estimated coefficients are highly correlated their
interpretations are confounded.

hth,

Kingsford Jones



>
> Thanks!
>
> Lep pozdrav / With regards,
>    Gregor Gorjanc
> ----------------------------------------------------------------------
> University of Ljubljana       PhD student
> Biotechnical Faculty          www: http://gregor.gorjanc.googlepages.com
> Department of Animal Science  blog: http://ggorjan.blogspot.com
> Groblje 3                     mail: gregor.gorjanc <at> bfro.uni-lj.si
> SI-1230 Domzale               fax: +386 (0)1 72 17 888
> Slovenia, Europe              tel: +386 (0)1 72 17 861
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>