[R] Order of terms in a model specification...

[Duncan Murdoch]

On 11/9/2005 3:48 PM, Oliver Lyttelton wrote:
> 
> 
> Hi,
> 
> Sorry for this one as its pretty basic but I've taken a look for info and
> couldn't find any...
> 
> My question is, does the order of main effect terms in a model specification
> have any impact on the model R fits or not. (in particular when using lm).
> ie
> 
> Can A~X+Y+Z lead to different results to A~Z+Y+X, and if so in what
> circumstances, and how much should I worry about it?
> 
> I believe this is an implementation detail as it depends on the way the
> fitting algorithm works, but it would be great to have a few lines to plug
> this gap in my knowledge...

Definitely yes, in the case of collinear terms.  For example,

 > X <- rnorm(10)
 > Y <- rnorm(10)
 > Z <- X
 > A <- rnorm(10)
 > lm(A ~ X+Y+Z)

Call:
lm(formula = A ~ X + Y + Z)

Coefficients:
(Intercept)            X            Y            Z
     -0.3474      -0.1166      -0.2203           NA

 > lm(A ~ Z+Y+X)

Call:
lm(formula = A ~ Z + Y + X)

Coefficients:
(Intercept)            Z            Y            X
     -0.3474      -0.1166      -0.2203           NA


In one case X gets a coefficient and Z doesn't, but the other is the 
opposite.

I suspect there would be differences due to rounding in other 
situations, and they might be noticeable in the case of near-collinearity.

Duncan Murdoch