[R] randomforest and AUC using 10 fold CV - Plotting results

[David martin]

Here is a snippet to show what i'm trying to do.

library(randomForest)
library(ROCR)
library(caret)

data(iris)
iris <- iris[(iris$Species != "setosa"),]

fit <- randomForest(factor(Species) ~ ., data=iris, ntree=50)
train.predict <- predict(fit,iris,type="prob")[,2]
plot(performance(prediction(train.predict,factor(iris$Species)),"tpr","fpr"),col 
= "red")
#As expected AUC is 1 because we are using the same dataset to validate
auc1 <- 
performance(prediction(train.predict,factor(iris$Species)),"auc")@y.values[[1]]
legend("bottomright",legend=c(paste("Random Forests 
(AUC=",formatC(auc1,digits=4,format="f"),")",sep="")),
                 col=c("red"), lty=1)


#Cross validation using 10 fold CV:
ctrl <- trainControl(method = "cv", classProbs = TRUE, summaryFunction = 
twoClassSummary)

set.seed(1)
rfEstimate <- train(factor(Species) ~ .,data = iris, method = "rf", 
metric = "ROC", tuneGrid = data.frame(.mtry = 2), trControl = ctrl)
rfEstimate


How can i plot the results from the cross validation on the previous ROC 
plot ?



thanks,
david