[R] ridge regression

Frank E Harrell Jr f.harrell at vanderbilt.edu
Wed Aug 19 16:37:11 CEST 2009


If you didn't post anonymously I would have made a suggestion.  Full 
names and affiliations should be given.

Frank


spime wrote:
> Dear all,
> 
> I considered an ordinary ridge regression problem. I followed three
> different ways:
> 1. estimate beta without any standardization
> 2. estimate standardized beta (standardizing X and y) and then again convert
> back  
> 3. estimate beta using lm.ridge() function
> 
> 
> X<-matrix(c(1,2,9,3,2,4,7,2,3,5,9,1),4,3)
> y<-t(as.matrix(cbind(2,3,4,5)))
> 
> n<-nrow(X)
> p<-ncol(X)
> 
> #Without standardization
> intercept <- rep(1,n)
> Xn <- cbind(intercept, X)
> K<-diag(c(0,rep(1.5,p)))
> beta1 <- solve(t(Xn)%*%Xn+K)%*%t(Xn)%*%y
> beta1
> 
> #with standardization
> ys<-scale(y)
> Xs<-scale(X)
> K<-diag(1.5,p)
> bs <- solve(t(Xs)%*%Xs+K)%*%t(Xs)%*%ys 
> b<- sd(y)*(bs/sd(X))
> intercept <- mean(y)-sum(as.matrix(colMeans(X))*b)
> beta2<-rbind(intercept,b)
> beta2
> 
> #Using lm.ridge function of MASS package
> beta3<-lm.ridge(y~X,lambda=1.5)
> 
> I'm getting three different outputs for using the above three different
> approaches, but which would been the same for all. 
> 
>> beta1
>                 [,1]
> intercept  3.4007944
>            0.3977462
>            0.2082025
>           -0.4829115
>> beta2
>                 [,1]
> intercept  3.3399855
>            0.1639469
>            0.0262021
>           -0.1228987
> 
>> beta3
>                      X1          X2          X3 
>  3.35158977  0.19460958  0.03152778 -0.15546775 
> 
> It will be very helpful to me if anybody can help me regarding why the
> outputs are coming different. 
> 
> regards.


-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University




More information about the R-help mailing list