[R] 632 estimator using boot package

[Jin Minming]

Thanks a lot. I have changed the calculation method by using optimism defined by Efron. The results from using boot and rms packages are quite close now.

Jim 


--- On Tue, 6/3/12, Frank Harrell <f.harrell at vanderbilt.edu> wrote:

> From: Frank Harrell <f.harrell at vanderbilt.edu>
> Subject: Re: [R] 632 estimator using boot package
> To: r-help at r-project.org
> Date: Tuesday, 6 March, 2012, 3:23
> Jim if I understand you, you are
> using the ordinary bootstrap instead of the
> Efron-Gong "optimism bootstrap".  I don't think the
> ordinary bootstrap is
> applicable here.
> Frank
> 
> Jin Minming wrote
> > 
> > Hello Frank,
> > 
> > Thanks for your advice. 
> > 
> > I also used the validation using rms package. The
> results from rms
> > indicate that the linear model used in my project is
> overfitted around 0.1
> > in terms of R2. But, the results from using ordinary
> boot method in boot
> > package suggest the mean of R2 is even higher than the
> original one.  Then
> > I guess maybe direclty using Boot package may not be a
> good idea for
> > assessing the overfitting in the regression. 
> > 
> > Jim
> > 
> > 
> > --- On Mon, 5/3/12, Frank Harrell
> <f.harrell@> wrote:
> > 
> >> From: Frank Harrell <f.harrell@>
> >> Subject: Re: [R] 632 estimator using boot package
> >> To: r-help@
> >> Date: Monday, 5 March, 2012, 17:06
> >> Bootstrapping does not leave one
> >> out.  As for .632 this is implemented in the
> >> rms package's validate and calibrate functions. 
> Note
> >> however that any
> >> claimed advantages of .632 over the ordinary
> optimism
> >> bootstrap seem to be a
> >> result only of the use of a discontinuous improper
> scoring
> >> role (proportion
> >> classified correctly).  The advantage may vanish
> when
> >> better scoring rules
> >> are used.  Simulations showing this may be found
> at
> >> http://biostat.mc.vanderbilt.edu/rms
> >> 
> >> Frank
> >> 
> >> Jin Minming wrote
> >> > 
> >> > Dear All,
> >> > 
> >> > Anyone has some idea how to implement 632
> estimator and
> >> leave-one out
> >> > bootstraping method by using boot package.  I
> know
> >> the bootstrap package
> >> > has this function, but it sounds not very
> flexible for
> >> my project. 
> >> > 
> >> > Thanks,
> >> > 
> >> > Jim
> >> > 
> >> >
> ______________________________________________
> >> > R-help@ mailing list
> >> > https://stat.ethz.ch/mailman/listinfo/r-help
> >> > PLEASE do read the posting guide
> >> > http://www.R-project.org/posting-guide.html
> >> > and provide commented, minimal,
> self-contained,
> >> reproducible code.
> >> > 
> >> 
> >> 
> >> -----
> >> Frank Harrell
> >> Department of Biostatistics, Vanderbilt University
> >> --
> >> View this message in context:
> >> http://r.789695.n4.nabble.com/632-estimator-using-boot-package-tp4446720p4446736.html
> >> Sent from the R help mailing list archive at
> Nabble.com.
> >> 
> >> ______________________________________________
> >> R-help@
> >> mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
> >> http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained,
> reproducible
> >> code.
> >>
> > 
> > ______________________________________________
> > R-help@ mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained,
> reproducible code.
> > 
> 
> 
> -----
> Frank Harrell
> Department of Biostatistics, Vanderbilt University
> --
> View this message in context: http://r.789695.n4.nabble.com/632-estimator-using-boot-package-tp4446720p4448633.html
> Sent from the R help mailing list archive at Nabble.com.
> 
> ______________________________________________
> R-help at r-project.org
> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible
> code.
>