[R] validate (rms package) using step instead of fastbw

[Ramon Diaz-Uriarte]

Dear All,

For logistic regression models: is it possible to use validate (rms
package) to compute bias-corrected AUC, but have variable selection
with AIC use step (or stepAIC, from MASS), instead of fastbw?


More details:

I've been using the validate function (in the rms package, by Frank
Harrell) to obtain, among other things, bootstrap bias-corrected
estimates of the AUC, when variable selection is carried out (using
AIC as criterion). validate calls predab.resample, which in turn calls
fastbw (from the Design package, by Harrell). fastbw " Performs a
slightly inefficient but numerically stable version of  fast backward
elimination on factors, using a method based on Lawless and Singhal
(1978). This method uses the fitted complete model (...)". However, I
am finding that the models returned by fastbw are much smaller than
those returned by stepAIC or step (a simple example is shown below),
probably because of the approximation and using the complete model.

I'd like to use step instead of fastbw. I think this can be done by
hacking predab.resample in a couple of places but I am wondering if
this is a bad idea (why?) or if I am reinventing the wheel.


Best,

R.


P.S. Simple example of fastbw compared to step:

library(MASS) ## for stepAIC and bwt data
example(birthwt)
library(rms)

bwt.glm <- glm(low ~ ., family = binomial, data = bwt)
bwt.lrm <- lrm(low ~ ., data = bwt)

step(bwt.glm)
## same as stepAIC(bwt.glm)

fastbw(bwt.lrm)



-- 
Ramon Diaz-Uriarte
Structural Biology and Biocomputing Programme
Spanish National Cancer Centre (CNIO)
http://ligarto.org/rdiaz
Phone: +34-91-732-8000 ext. 3019