Hi all,

This is an old topic but I would want to have your opinion about what  
I have found until now.

The question is: "How to compare several Cox models by resampling  
(e.g. cross-validation or bootstrap) ? Which measure ?"
I would want to have such measures to do model selection (select the  
best model and indirectly select the best variables in a robust way).  
So the models may be nested or not.

Here are the different alternatives:

1. cross-validated partial likelihood (CVPL) as defined in J. Huang  
and D. Harrington, Penalized Partial Likelihood Regression for Right- 
Censored Data with Bootstrap Selection of the Penalty Parameter,  
2002, Biometrics.

2.  bootstrap estimated mean squared estimation error (BMS) as  
defined in the previously cited article. Unfortunately this measure  
does not seem to be adequate to compare models with different sets of  
variables.

3. partial log likelihood as return by the coxph or cph functions. I  
have written a small function to compute the partial log likelihood  
from the beta's of a fitted Cox model given a test test.

4. Dxy using the validate.cph function from the Design package.  
According to the documentation, I have to use the Dxy index because  
my Cox models are stratified (it is a characteristic of my dataset,  
patients coming from different hospitals). Unfortunately, this  
function does not return the details about each iteration (it just  
prints them to the screen), but a small modification goes through  
this. There are 2 methods: crossvalidation and bootstrap.

For my dataset, the only acceptable method is the validate.cph with  
bootstrap. All the other methods give me "strange" results (I mean,  
not expected but I have not perform a complete simulation study).

What are your opinions/experience ? If somebody wants to test them in  
a dataset, I can provide you such an example with my R functions.

Thanks in advance for your precious help.

Best regards

---
Benjamin Haibe-Kains
	[http://www.ulb.ac.be/di/map/bhaibeka/]
bhaibeka@ulb.ac.be

Machine Learning Group (ULB)
	[http://www.ulb.ac.be/di/mlg/]

MicroArray Unit (IJB)
	[http://www.bordet.be/servmed/array/]



	[[alternative HTML version deleted]]

