[Rd] lsfit w/ rank-deficient x

[Vadim Ogranovich]

Actually, the correct permutation is given by the inverse of qr$pivot:

foo$coefficients[foo$qr$pivot] <- foo$coefficients

Here foo is an object returned by lsfit, see below.



-----Original Message-----
From: Vadim Ogranovich
Sent: Friday, March 13, 2009 5:25 PM
To: 'r-devel at r-project.org'
Subject: lsfit w/ rank-deficient x

Dear R-devel,

It seems that lsfit incorrectly reports coefficients when the input matrix 'x' is rank-deficient, see the example below:

## here values of 'b' and 'c' are incorrectly swapped
> x <- cbind(a=rnorm(100), b=0, c=rnorm(100)); y <- rnorm(100); lsfit(x, y)$coef
 Intercept          a          b          c
-0.0227787  0.1042860 -0.1729261  0.0000000
Warning message:
In lsfit(x, y) : 'X' matrix was collinear

## correct values
> lsfit(x[,-2], y)$coef
 Intercept          a          c
-0.0227787  0.1042860 -0.1729261


I looked inside the lsfit code and it appears that even though rank-deficiency is detected there is no attempt to patch the coefficients. Why is that?

Taking clues from the code it appears that the following trick might do the work:

> foo <- lsfit(x, y)
Warning message:
In lsfit(x, y) : 'X' matrix was collinear
> structure(foo$coefficients[foo$qr$pivot], names=names(foo$coefficients))
  Intercept           a           b           c
 0.14857345 -0.07473099  0.00000000  0.12835155


Is this reliable or there are cases when it may fail?

Thanks,
Vadim

P.S.

> version
               _
platform       i386-pc-mingw32
arch           i386
os             mingw32
system         i386, mingw32
status
major          2
minor          7.1
year           2008
month          06
day            23
svn rev        45970
language       R
version.string R version 2.7.1 (2008-06-23)
>

Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments.  Email transmission cannot be guaranteed to be secure or error-free.  Jump Trading, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments.  This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.