# [R] different L2 regularization behavior between lrm, glmnet, and penalized?

Robert V (Bob) Sasseen sasseen at ai.sri.com
Wed Oct 14 19:57:35 CEST 2009

```   The following R code using different packages gives the same results for a
simple logistic regression without regularization, but different results
with regularization. This may just be a matter of different scaling of the
regularization parameters, but if anyone familiar with these packages has
insight into why the results differ, I'd appreciate hearing about it. I'm
new to R. Thanks. (Version info below. Same results on Windows and Solaris
8, except that I haven't gotten glmnet to compile on the latter.)
Robert V (Bob) Sasseen
sasseen at ai.sri.com
> # Some x values (predictive var).
> x = c(-2, -2, -2, -2, -1, -1, -1, 2, 2, 2, 3, 3, 3, 3)
> # Some z values--the observed outcome.
> # Effect is that for
> # x = -2, p = 1/4;
> # x = -1, p = 1/3;
> # x =  2, p = 2/3;
> # x =  3, p = 3/4.
> z = c(0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1)
> library(Design)
> g = lrm(z ~ x)
> g\$coefficients
Intercept          x
-0.2224842  0.4449685
> g = lrm(z ~ x, penalty = 1)
> g\$coefficients
Intercept          x
-0.1620727  0.3241454
> library(glmnet)
> g = glmnet(cbind(x), cbind(1-z, z), family = "binomial", lambda = 0,
standardize = FALSE)
> coef(g)
1
-0.2224843
x  0.4449687
> g = glmnet(cbind(x), cbind(1-z, z), family = "binomial", lambda = 1, alpha
= 0, standardize = FALSE)
> coef(g)
1
-0.1098361
x  0.2196721
> library(penalized)
> fit = penalized(z ~ x)
> coefficients(fit, "all")
(Intercept)           x
-0.2224843   0.4449687
> fit = penalized(z ~ x, lambda2 = 1)
> coefficients(fit, "all")
(Intercept)           x
-0.2060658   0.4121315
> sessionInfo()
R version 2.9.2 (2009-08-24)
i386-pc-mingw32
locale:
LC_COLLATE=English_United          States.1252;LC_CTYPE=English_United
States.1252;LC_MONETARY=English_United
States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
attached base packages:
 splines   stats     graphics  grDevices utils     datasets  methods
 base
other attached packages:
     penalized_0.9-26      glmnet_1.1-3          Matrix_0.999375-30
lattice_0.17-25
 Design_2.3-0       Hmisc_3.7-0        survival_2.35-4
loaded via a namespace (and not attached):
 cluster_1.12.0 grid_2.9.2

References

1. mailto:sasseen at ai.sri.com

```