Hi Mark,

I also have been suspicious of filtering out low-variance genes. 
There are 1-2 issues, depending on how you calculate the test 
statistic. The first is the possible distortion of FDR calculations. 
On page 234 of the BioC book you mention, Scholtens & Heydebreck 
state: "If the truly differentially expressed genes are 
overrepresented among those selected in the filtering step, the FDR 
associated with a certain threshold of the test statistic will be 
lowered due to filtering."  I believe this is what your colleague was 
complaining about, as I can't see how filtering based on variance 
could do anything except overrepresent DEG. But - is this a good or bad thing?

The second issue may not matter depending on how you calculate the 
test statistic. If you use limma, which uses a Bayesian shrinkage 
based on the variance of all genes, filtering out low-variance genes 
will lead to a higher average gene variance, and hence less 
shrinkage. Scholtens & Heydebreck also point this out on pg. 234: 
"Also concerning the variability across samples, a higher overall 
variance of the differentially expressed genes may be expected, 
because their between-class variance adds to their within-class 
variance."  I've looked at this in some of my data sets, and the t 
values calculated by eBayes() after filtering are generally lower 
than they were before filtering. Now, the boost to FDR correction for 
a fewer number of genes tends to cancel out the lower p-values, but not always.

I also would be interested in any theoretical papers on this that 
confirm or rebuke my intuitions. I myself have no problem doing a 
conservative filtering to remove genes that likely aren't expressed 
in any of the samples, but I don't filter based on variance because I 
use the Bayesian correction in limma. There was a lengthy exchange on 
this top in Feb 2006 (subject: Invalid fold-filter). Robert Gentleman 
https://stat.ethz.ch/pipermail/bioconductor/2006-February/011988.html 
said you can do simulations to convince yourself that filtering on 
variance doesn't really bias the results, at least in regards to the 
first issue I mention. The effects on the Bayesian shrinkage I don't 
think have been addressed...

Cheers,
Jenny

At 11:36 PM 4/2/2007, Mark W Kimpel wrote:
>I have been using what I consider to be non-biased filtering of
>low-variance genes using the method described in "Bioinformatics and
>Computational Biology Solutions using R and Bioconductor", R. Gentleman,
>et al., page 233 for some time and have recently run into some
>resistance from a colleague who claims that this type of filtering
>distorts FDR calculations because it introduces bias. His reasoning is
>that, since this method tends to filter out genes with higher p values
>and/or lower fold changes, that it is sort of a sneaky way of
>accomplishing just that. Of course, filtering by phenotype does
>introduce bias, but in this case I believe that by filtering based on
>the a priori assumption that we just aren't that interested in low
>variance genes for biologic reasons (even if statistically significant
>they will have very low fold changes and thus be of questionable
>meaning) that we aren't violating the statistical underpinnings of the
>analysis.
>
>I need some help in justifying this filtering step. Does anyone know of
>a peer-reviewed reference that gives a theoretical justification for its
>use of of any empiric experiments that show that it is legit?
>
>Thanks,
>Mark
>--
>Mark W. Kimpel MD
>Neuroinformatics
>Department of Psychiatry
>Indiana University School of Medicine
>
>_______________________________________________
>Bioconductor mailing list
>Bioconductor@stat.math.ethz.ch
>https://stat.ethz.ch/mailman/listinfo/bioconductor
>Search the archives: 
>http://news.gmane.org/gmane.science.biology.informatics.conductor

Jenny Drnevich, Ph.D.

Functional Genomics Bioinformatics Specialist
W.M. Keck Center for Comparative and Functional Genomics
Roy J. Carver Biotechnology Center
University of Illinois, Urbana-Champaign

330 ERML
1201 W. Gregory Dr.
Urbana, IL 61801
USA

ph: 217-244-7355
fax: 217-265-5066
e-mail: drnevich@uiuc.edu 
	[[alternative HTML version deleted]]

