[BioC] adjustment for dependent hypotheses when pairwise testing

[Gordon Smyth]

A good question. The book on microarray expression data by McLachlan, 
Do and Ambroise contains a review of FDR methods in Chapter 5, but a 
more detailed review is much needed.

BTW, Yoav Benjamini has shown me that one can substitute a particular 
first order approximation to pi0 into the BH method without harming 
the ability of the method to work for small numbers of genes. So far, 
I haven't been motivated enough to do that because the improvement is 
usually relatively small for microarray data sets.

Cheers
Gordon

At 01:13 PM 16/01/2006, Naomi Altman wrote:
>This discussion convinced me to read through my collection of FDR 
>papers, and I am finding it heavy going.  Is there a review paper 
>somewhere that goes through these methods + Storey et al (and maybe 
>the Simes and Bonferroni FWER methods) and explains their 
>applicability to the types of dependence structures typically found 
>in e.g. factorial ANOVA where we may be testing main effects and 
>interactions, or 1-way ANOVA with all pairwise comparisons, or 
>eBayes tests like limma, where dependence is induced by the 
>denominator adjustment?
>
>Thanks,
>Naomi
>
>
>At 04:32 PM 1/14/2006, Gordon K Smyth wrote:
>>Actually I don't think the number of genes is a problem.  Neither 
>>BH nor BY use an estimate of pi0
>>(the true proportion of hull hypotheses).  Whenever pi0 would 
>>appear as a multiplier in the
>>formulae, they use 1 instead.  The methods are therefore somewhat 
>>conservative if pi0 is not
>>large, but they continue to control the FDR at below the requested level.
>>
>>This means that BY gives rigorous control of expected FDR for any 
>>dependence structure and any
>>number of genes, given only that the p-value distribution is valid 
>>under the null hypothesis.  BH
>>is the same except that it is theoretically valid only for certain 
>>dependence structures (positive
>>regression dependence).
>>
>>I think that Rebecca's question is about the particular (negative) 
>>dependence structure which is
>>found between the set of pairwise contrasts for a particular 
>>gene.  Yes, I think this does
>>theoretically invalidate BH.  However in practical situations 
>>you'll actually find the main
>>problem to be conserativism rather than the other way around 
>>because there are more pairwise
>>comparisons than there are independent comparisons to be made.  So, 
>>personally, this wouldn't make
>>me move to BY.
>>
>>Hope this helps
>>Gordon
>>
>> > Date: Fri, 13 Jan 2006 16:57:28 -0500
>> > From: Naomi Altman <naomi at stat.psu.edu>
>> > Subject: Re: [BioC] adjustment for dependent hypotheses when pairwise
>> >       testing
>> > To: pmt1rew at leeds.ac.uk,      "bioconductor at stat.math.ethz.ch"
>> >       <bioconductor at stat.math.ethz.ch>
>> >
>> > I do not think you will be able to use these methods with such a
>> > small number of genes.  A large number of genes are required to 
>> estimate pi0.
>> >
>> > --Naomi
>> >
>> > At 07:52 AM 1/13/2006, pmt1rew at leeds.ac.uk wrote:
>> >>Dear all,
>> >>
>> >>I data for 13 genes with the samples from 4 different tissue
>> >>types.  Would like
>> >>to compare the normal means of the different tissue types using pairwise
>> >>testing with an adjustment for fdr.
>> >>
>> >>Having considered the Benjamini and Yekutieli reference on the 
>> p.adjust help
>> >>page I am unsure as to whether my dependency structure satisfies 
>> the criteria
>> >>to just use the regular "BH" adjustment or if  the "BY" 
>> adjustment would be
>> >>more appropriate?
>> >>
>> >>Any suggestions for me please?
>> >>
>> >>Many thanks
>> >>
>> >>Rebecca Walls
>> >>
>> > Naomi S. Altman                                814-865-3791 (voice)
>> > Associate Professor
>> > Dept. of Statistics                              814-863-7114 (fax)
>> > Penn State University                         814-865-1348 (Statistics)
>> > University Park, PA 16802-2111
>
>Naomi S. Altman                                814-865-3791 (voice)
>Associate Professor
>Dept. of Statistics                              814-863-7114 (fax)
>Penn State University                         814-865-1348 (Statistics)
>University Park, PA 16802-2111
>