[BioC] False Discovery Rate Questions

[James W. MacDonald]

Hi Sally,

Sally wrote:
> We had a discussion in our lab about false discovery rate correction.
> Most people felt a little disappointed with their microarray data
> after in many cases a long hard slog with the lab process.  The FDR
> (BY, BH etc. etc.) knocked their un-adjusted gene lists into
> very,very short lists of differentially expressed genes.  I work on
> disease pathology and thought microarrays would provide me with a
> constellation of clues.  Instead I find for some time periods I have
> none.  This is with a 17,500 cDNA custom chip.
> 
> It surprised me that many knew of papers published without applying
> an FDR adjustment.  I had just assumed that for a proper statistical
> analysis had to have one.

If you are looking for a proper statistical analysis, you better stop 
using microarrays ;-D

I think people put way too much stock in multiplicity adjustments. The 
goal of these adjustments is to limit the number of false positives in a 
set of 'significant' genes. However, to my knowledge all multiplicity 
adjustments are monotonic, so the base ranking of the genes will never 
change.

So let's say you figure you can reasonably validate 50 genes. If you do 
the FDR adjustment and end up with 5000 genes at a 5% FDR, does that 
really help you? You will either take the top 50 and validate, or scan 
the top of the list for 'interesting' genes (I call this an 
eyeballometric analysis) and then validate those. Or maybe you will try 
to subset the significant genes by some interesting process or pathway. 
Regardless, the fact that you have a 5% FDR will likely be irrelevant.

On the other hand, let's say you have no genes at a 5% FDR. This doesn't 
mean there are no significant genes; instead it means that you don't 
have the power to detect differences. But the genes at the top of the 
list are (according to the data in hand) the most likely to validate, so 
you could just take the top 50 and validate. Or you could not adjust and 
take however many have an unadjusted p-value of 0.005 or whatever.

Best,

Jim


> 
> I wonder what people are doing regarding using/not using an FDR?
> 
> What FDR corrections are people using?
> 
> What newer alternatives are there that are less conservative?
> 
> Sally Goldes [[alternative HTML version deleted]]
> 
> _______________________________________________ Bioconductor mailing
> list Bioconductor at stat.math.ethz.ch 
> https://stat.ethz.ch/mailman/listinfo/bioconductor Search the
> archives:
> http://news.gmane.org/gmane.science.biology.informatics.conductor

-- 
James W. MacDonald, M.S.
Biostatistician
Affymetrix and cDNA Microarray Core
University of Michigan Cancer Center
1500 E. Medical Center Drive
7410 CCGC
Ann Arbor MI 48109
734-647-5623