[R] How to add 95% confidence intervals in the calibration plot?

[Frank E Harrell Jr]

zhu yao wrote:
> Thanks for your reply.
> 
> Actually, I'm confused about the results in the article "Postoperative 
> nomogram for survival of patients with retroperitoneal sarcoma treated 
> with curative intent"
> http://annonc.oxfordjournals.org/cgi/content/abstract/mdp298v1
> 
> It stated as:
> 
> nomogram model
> The Cox model was used as the basis for the nomogram
> (Table 2). Figure 2 depicts the final nomogram and portrays the
> association between each variable and survival based on the
> scoring system derived from this analysis. The concordance
> index (discrimination) after internal validation with 200
> bootstrapping resamples was 0.73 (95% CI 0.71–0.75).
> Similarly, Figure 3 illustrates the calibration of the nomogram
> before and after internal validation with bootstrapping samples.
> Calibration was excellent with observed outcomes always
> within 95% CI of the predicted survival probability.

Figure 3 is provided by the Design package without modification.  As I 
stated before it does provide those CIs for survival models.  I guess 
that the CI for the c-index was obtained without bootstrap validation 
using the Hmisc package's rcorr.cens function (and Dxy=2*(C-.5)) or by 
using an approximate bootstrap analysis they programmed.

Note that in the abstract the authors wrongly used the confidence 
intervals in Fig 3 to conclude excellent validation of the model.  Their 
conclusion can arise from just having large confidence intervals.

Frank

> 
> Figure 3 is
> http://i3.6.cn/cvbnm/a9/c8/8b/c01aad248a0b4ae6ef677600614bd4fa.jpg
> 
> 
> 2009/7/26 Frank E Harrell Jr <f.harrell at vanderbilt.edu 
> <mailto:f.harrell at vanderbilt.edu>>
>  >
>  > zhu yao wrote:
>  >>
>  >> Dear experts:
>  >>
>  >> I am a newbie to R. Recently, I try to make prediction models with R 
> and the
>  >> Design library.
>  >> I have read Prof. Harrell's excellent book. But I did not quite 
> understand.
>  >> I have two problems about the validation and calibration of prediction
>  >> models:
>  >> 1. Can someone explain the results outputted by the validate() 
> function? How
>  >> to get 95% of c-value of validate?
>  >
>  > validate does not provide that confidence interval, unfortunately.
>  >
>  >> 2. How to add 95% ci in the calibration plot?
>  >
>  > That is not provided except for survival models.
>  >
>  > Next time please include your code so we can see what model you are 
> using.
>  >
>  > Thanks
>  > Frank
>  >
>  >>
>  >> Yao Zhu
>  >> Department of Urology
>  >> Fudan University Shanghai Cancer Center
>  >>
>  >>        [[alternative HTML version deleted]]
>  >>
>  >> ______________________________________________
>  >> R-help at r-project.org <mailto:R-help at r-project.org> mailing list
>  >> https://stat.ethz.ch/mailman/listinfo/r-help
>  >> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
>  >> and provide commented, minimal, self-contained, reproducible code.
>  >>
>  >