[R] Jagged ROC curves?

[Marc Schwartz]

Hi Brian,

Your underlying dataset for the ROC curve only has 4 unique values for specificity, even though there are 23 elements in the vector, hence the step function nature of the first plot.

The default smoothing in the smooth() function is "binormal". You might try one of the other smoothing options to see the result and whether they make visual sense.

In the absence of smoothing, there will always be a step function, but some will look "more smooth" than others depending upon your data and how the thresholds are defined for the underlying metrics. The more unique thresholds there are over the range, the smoother the step function will look.

Regards,

Marc


> On Jun 26, 2017, at 1:07 PM, Brian Smith <bsmith030465 at gmail.com> wrote:
> 
> Hi Mark,
> 
> Thanks for the reply.
> 
> For the attached two png files (test_roc.png & test_roc_smooth.png)
> 
> 1. Using 'plot' function:
> 
> plot(c(1,0),c(0,1), type='l', lty=3, xlim=c(1.01,-0.01), ylim=c(-0.01,1.01), xaxs='i', yaxs='i', ylab='', xlab='')
> plot(roc_1,col="brown3", lwd=2, add=T, lty=1)
> 
> 2. Using the 'smooth' function:
> 
> plot(c(1,0),c(0,1), type='l', lty=3, xlim=c(1.01,-0.01), ylim=c(-0.01,1.01), xaxs='i', yaxs='i', ylab='', xlab='')
> plot(smooth(roc_1),col="brown3", lwd=2, add=T, lty=1)
> 
> 
> I guess most ROCs that I've seen are somewhere in between, i.e. they have a little jaggedness, but not as much as in plot #1 above.<test_roc.png><test_roc_smooth.png>
> 
> thanks!
> 
> Pankaj
> 
> On Mon, Jun 26, 2017 at 12:59 PM, Marc Schwartz <marc_schwartz at me.com <mailto:marc_schwartz at me.com>> wrote:
> 
> > On Jun 26, 2017, at 11:40 AM, Brian Smith <bsmith030465 at gmail.com <mailto:bsmith030465 at gmail.com>> wrote:
> >
> > Hi,
> >
> > I was trying to draw some ROC curves (prediction of case/control status),
> > but seem to be getting a somewhat jagged plot. Can I do something that
> > would 'smooth' it somewhat? Most roc curves seem to have many incremental
> > changes (in x and y directions), but my plot only has 4 or 5 steps even
> > though there are 22 data points. Should I be doing something differently?
> >
> > How can I provide a URL/attachment for my plot? Not sure if I can provide
> > reproducible code, but here is some pseudocode, let me know if you'd like
> > more details:
> >
> > #####
> > ## generate roc and auc values
> > #####
> > library(pROC)
> > library(AUCRF)
> >
> > getROC <- function(d1train,d1test){
> > my_model <- AUCRF(formula= status ~ ., data=d1train,
> > ranking='MDA',ntree=1000,pdel=0.05)
> >  my_opt_model <- my_model$RFopt
> >
> >  my_probs <- predict(my_opt_model, d1test, type = 'prob')
> >  my_roc <- roc(d1test[,resp_col] ~ my_probs[,2])
> >  aucval <- round(as.numeric(my_roc$auc),4)
> > return(my_roc)
> > }
> >
> >
> > roc_1 <- getROC(dat1,dat1test)
> > plot.roc(roc_1,col="brown3")
> >
> >
> >> roc_1
> >
> > Call:
> > roc.formula(formula = d1test[, resp_col] ~ ibd_probs[, 2])
> >
> > Data: ibd_probs[, 2] in 3 controls (d1test[, resp_col] 0) < 19 cases
> > (d1test[, resp_col] 1).
> > Area under the curve: 0.8596
> >
> >
> >> roc_1$sensitivities
> > [1] 1.00000000 0.94736842 0.94736842 0.94736842 0.89473684 0.84210526
> > 0.78947368 0.73684211 0.68421053 0.68421053
> > [11] 0.63157895 0.57894737 0.52631579 0.47368421 0.42105263 0.36842105
> > 0.31578947 0.26315789 0.21052632 0.15789474
> > [21] 0.10526316 0.05263158 0.00000000
> >
> >
> >> roc_1$specificities
> > [1] 0.0000000 0.0000000 0.3333333 0.6666667 0.6666667 0.6666667 0.6666667
> > 0.6666667 0.6666667 1.0000000 1.0000000
> > [12] 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000
> > 1.0000000 1.0000000 1.0000000 1.0000000
> > [23] 1.0000000
> >
> >
> > many thanks!
> 
> 
> ROC curves are typically step functions of some nature, depending upon your thresholds, so the default behavior is not going to be smoothed.
> 
> I am not sure how they (AUCRF and pROC) may interact, but look at the ?smooth function in the latter package to see if it might help.
> 
> To your second point, if your plot is a png/jpg file, you could attach it to your post here, if that was your desire. Otherwise, you could post it to a cloud based repository, like Dropbox, and provide the URL for public sharing here. The R lists support limited binary attachment types and png/jpg/pdf/ps are supported.
> 
> Regards,
> 
> Marc Schwartz
> 
> 


	[[alternative HTML version deleted]]