Dear Roger,

You may want to consult the paper cited below.

Best regards,
Martin





************************************************************

        Bioinformatics. 2004 Feb 12;20(3):374-80 



Is cross-validation valid for small-sample microarray classification?
Braga-Neto UM, Dougherty ER.
Section of Clinical Cancer Genetics, University of Texas MD Anderson 
Cancer Center, Houston, TX, USA.
MOTIVATION: Microarray classification typically possesses two striking 
attributes: (1) classifier design and error estimation are based on 
remarkably small samples and (2) cross-validation error estimation is 
employed in the majority of the papers. Thus, it is necessary to have a 
quantifiable understanding of the behavior of cross-validation in the 
context of very small samples. RESULTS: An extensive simulation study has 
been performed comparing cross-validation, resubstitution and bootstrap 
estimation for three popular classification rules-linear discriminant 
analysis, 3-nearest-neighbor and decision trees (CART)-using both 
synthetic and real breast-cancer patient data. Comparison is via the 
distribution of differences between the estimated and true errors. Various 
statistics for the deviation distribution have been computed: mean (for 
estimator bias), variance (for estimator precision), root-mean square 
error (for composition of bias and variance) and quartile ranges, 
including outlier behavior. In general, while cross-validation error 
estimation is much less biased than resubstitution, it displays excessive 
variance, which makes individual estimates unreliable for small samples. 
Bootstrap methods provide improved performance relative to variance, but 
at a high computational cost and often with increased bias (albeit, much 
less than with resubstitution).






"Roger Liu" <zrl1974@gmail.com> 
Sent by: bioconductor-bounces@stat.math.ethz.ch
31.08.2007 17:18

To
bioconductor <bioconductor@stat.math.ethz.ch>
cc

Subject
[BioC] small sample size,microarray, cross-validation






Dear Friends,

I am trying to use some supervised method to identify the genes which 
permit
the separation of two patients conditions.

Due to the extreme difficulty, I can only have the test set (about 17
patients), could I just use these 17 data sets as training set as well as
test, such as using leave one out cross validation?
I know its result may be optimistic, but I am just wondering:

1. if you have any good idea about  this  small sample size issue in
classification analysis.
2. is the cross-validation a legitimate method to support that I have
identified the genes which can separate the groups as different 
conditions?
we try to get this result published, will it be an OK as using this
cross-validation?
3. what packages, methods, will you recommend for this issue?

Thanks.


Regards,
-Roger

                 [[alternative HTML version deleted]]

_______________________________________________
Bioconductor mailing list
Bioconductor@stat.math.ethz.ch
https://stat.ethz.ch/mailman/listinfo/bioconductor
Search the archives: 
http://news.gmane.org/gmane.science.biology.informatics.conductor


	[[alternative HTML version deleted]]

