[R] Package for .632 (and .632+) bootstrap and the cross-validation of ROC Parameters

[spime]

Suppose I have

Training data: my.train
Testing data: my.test

I want to calculate bootstrap error rate for logistic model. My wrapper
function for prediction

pred.glm <- function(object, newdata) {
        ret <- as.factor(ifelse(predict.glm(object, newdata,
type='response') < 0.4, 0, 1))
        return(ret)
        }

But i thing i cant understand if i want to calculate misclassification error
for my testing data what will be in my data in the following formula.

errorest(RES ~., data=???, model=glm, estimator="boot", predict=pred.glm, 
       est.para=control.errorest(nboot = 10))

Using my.test got following error,

Error in predict(mymodel, newdata = outbootdata) : 
        unused argument(s) (newdata = list(RES = c(1, 0, 0, 0, 1, 0, 0, 0,
0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1,
1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0,
0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1,
0), CAT01 = c(4, 4, 2, 4, 4, 4, 4, 4, 4, 2, 1, 2, 2, 4, 4, 4, 1, 1, 2, 2, 1,
4, 1, 4, 1, 4, 2, 4, 1, 4, 2, 3, 1, 1, 3, 3, 4, 2, 4, 2, 1, 2, 2, 1, 1, 
> 

please reply...






Frank E Harrell Jr wrote:
> 
> spime wrote:
>> 
>> Hi users,
>> 
>> I need to calculate .632 (and .632+) bootstrap and the cross-validation
>> of
>> area under curve (AUC) to compare my models. Is there any package for the
>> same. I know about 'ipred' and using it i can calculate misclassification
>> errors. 
>> 
>> Please help. It's urgent. 
> 
> See the validate* functions in the Design package.
> 
> Note that some simulations (see http://biostat.mc.vanderbilt.edu/rms) 
> indicate that the advantages of .632 and .632+ over the ordinary 
> bootstrap are highly dependent on the choice of the accuracy measure 
> being validated.  The bootstrap variants seem to have advantages mainly 
> if an improper, inefficient, discontinuous scoring rule such as the 
> percent classified correct is used.
> 
> -- 
> Frank E Harrell Jr   Professor and Chair           School of Medicine
>                       Department of Biostatistics   Vanderbilt University
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 

-- 
View this message in context: http://www.nabble.com/Package-for-.632-%28and-.632%2B%29-bootstrap-and-the-cross-validation-of-ROC-Parameters-tf4068544.html#a11578129
Sent from the R help mailing list archive at Nabble.com.