[R-sig-ME] Low rank fixed effect design matrix LMER package

[Ben Bolker]

  The place to start looking is line 222 of 
https://github.com/lme4/lme4/blob/master/R/modular.R . We call the 
Matrix::rankMatrix() and stats::qr() functions with a default tolerance 
of 1e-7.  The comment in the code at that point specifies

## Perform the qr-decomposition of X using LINPACK method,
## as we need the "good" pivots (and the same as lm()):
## FIXME: strongly prefer rankMatrix(X, method= "qr.R")

   Hope that helps.

   Ben Bolker


On 16-02-10 08:39 PM, suraj keshri wrote:
> Hi,
>
> I'm trying to implement some pieces of LMER algorithm in Python. I'm facing
> low rank fixed effect design matrix issue. One way to take care of low rank
> is to detect linearly dependent columns using ideas from SVD/QR
> decomposition and remove it. My design matrix is very big and sparse.
> Therefore, instead of doing SVD of the design matrix (X), I do SVD of X^T *
> X. However, one has to decide a threshold for the singular values. When I
> look at the singular values of X^T*X, the range is very large (1e+12 to
> 1e-4). In this situation, how does one decide the threshold so that X^T*X
> is invertible (after removing linearly dependent columns)? How does LMER
> package solve this problem?
>
> Thank you so much for any help!!
>
>
> Suraj
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>