[R] Ridge regression and mixed models

[Dimitri Liakhovitski]

Curious - what would be the purpose of this regression?

On Mon, Oct 4, 2010 at 4:39 PM, harezlak at post.harvard.edu
<jarek67h at yahoo.com> wrote:
> Dear R users,
>  An equivalence between linear mixed model formulation and penalized regression
> models (including the ridge regression and penalized regression splines) has
> proven to be very useful in many aspects. Examples include the use of the lme()
> function in the library(nlme) to fit smooth models including the estimation of a
> smoothing parameter using REML. My question concerns the use of the linear mixed
> model software to fit a ridge regression with the number of columns in the
> design matrix X (p) exceeding the number of observations (n). Has anybody in the
> R community implemented the LME-like approach with estimation of the variance
> components using REML to find the coefficient estimates (BLUEs) and predictors
> (BLUPs) in the ridge regression problem in the "p > n"  setting?
>
> Sample code below summarizes my problem:
> ####################################################
> version$version.string
> # [1] "R version 2.11.1 (2010-05-31)"
>
> library(nlme)
>
> # DATA generation:
> dim <- 200
> n <- 50
> XX <- matrix(rnorm(dim*n, 0, 0.1), ncol=dim, nrow=n)
> beta <- matrix(c(rep(1, 40), rep(2,20), rep(0,140)), ncol=1)
> Y <- XX %*% beta + rnorm(n)
>
> # MODEL fit:
> dummyId <- factor(rep(1,n))
> Z.block <- list(dummyId=pdIdent(~-1+XX))
> data.fr <- data.frame(Y,XX)
> fit <- lme(Y~1,
>        data=data.fr,
>        random=Z.block)
>
> # ERROR:
> Warning message:
> In lme.formula(Y ~ 1, data = data.fr, random = Z.block) :
>  Fewer observations than random effects in all level 1 groups
> #############################################################
>
> Thank you in advance,
> Jarek  Harezlak
>
>
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Dimitri Liakhovitski
Ninah Consulting
www.ninah.com