[R] Leave one out Cross validation (LOO)
Frank E Harrell Jr
f.harrell at vanderbilt.edu
Wed Feb 25 14:38:12 CET 2009
Alex Roy wrote:
> Dear Frank,
> Thanks for your comments. But in my situation, I do
> not have any future data and I want to calculate Mean Square Error for
> prediction on future data. So, is it not it a good idea to go for LOO?
With resampling you should be able to estimate any parameter including
sigma. The Design package's validate.ols function can estimate sigma
using the bootstrap or c-v, penalizing for backward stepdown variable
selection, although I have found some counter-intuitive estimates of
sigma using Efron's optimism bootstrap.
> On Tue, Feb 24, 2009 at 7:15 PM, Frank E Harrell Jr
> <f.harrell at vanderbilt.edu <mailto:f.harrell at vanderbilt.edu>> wrote:
> Alex Roy wrote:
> Dear R user,
> I am working with LOO. Can any one who is
> with leave one out cross validation (LOO) could send me the code?
> Thanks in advance
> I don't think that LOO adequately penalizes for model uncertainty.
> I recommend the bootstrap or 50 repeats of 10-fold
> cross-validation. See for example the validate and calibrate
> functions in the R Design package.
More information about the R-help