[BioC] adjustment for dependent hypotheses when pairwise testing

[Naomi Altman]

This discussion convinced me to read through my collection of FDR 
papers, and I am finding it heavy going.  Is there a review paper 
somewhere that goes through these methods + Storey et al (and maybe 
the Simes and Bonferroni FWER methods) and explains their 
applicability to the types of dependence structures typically found 
in e.g. factorial ANOVA where we may be testing main effects and 
interactions, or 1-way ANOVA with all pairwise comparisons, or eBayes 
tests like limma, where dependence is induced by the denominator adjustment?

Thanks,
Naomi


At 04:32 PM 1/14/2006, Gordon K Smyth wrote:
>Actually I don't think the number of genes is a problem.  Neither BH 
>nor BY use an estimate of pi0
>(the true proportion of hull hypotheses).  Whenever pi0 would appear 
>as a multiplier in the
>formulae, they use 1 instead.  The methods are therefore somewhat 
>conservative if pi0 is not
>large, but they continue to control the FDR at below the requested level.
>
>This means that BY gives rigorous control of expected FDR for any 
>dependence structure and any
>number of genes, given only that the p-value distribution is valid 
>under the null hypothesis.  BH
>is the same except that it is theoretically valid only for certain 
>dependence structures (positive
>regression dependence).
>
>I think that Rebecca's question is about the particular (negative) 
>dependence structure which is
>found between the set of pairwise contrasts for a particular 
>gene.  Yes, I think this does
>theoretically invalidate BH.  However in practical situations you'll 
>actually find the main
>problem to be conserativism rather than the other way around because 
>there are more pairwise
>comparisons than there are independent comparisons to be made.  So, 
>personally, this wouldn't make
>me move to BY.
>
>Hope this helps
>Gordon
>
> > Date: Fri, 13 Jan 2006 16:57:28 -0500
> > From: Naomi Altman <naomi at stat.psu.edu>
> > Subject: Re: [BioC] adjustment for dependent hypotheses when pairwise
> >       testing
> > To: pmt1rew at leeds.ac.uk,      "bioconductor at stat.math.ethz.ch"
> >       <bioconductor at stat.math.ethz.ch>
> >
> > I do not think you will be able to use these methods with such a
> > small number of genes.  A large number of genes are required to 
> estimate pi0.
> >
> > --Naomi
> >
> > At 07:52 AM 1/13/2006, pmt1rew at leeds.ac.uk wrote:
> >>Dear all,
> >>
> >>I data for 13 genes with the samples from 4 different tissue
> >>types.  Would like
> >>to compare the normal means of the different tissue types using pairwise
> >>testing with an adjustment for fdr.
> >>
> >>Having considered the Benjamini and Yekutieli reference on the 
> p.adjust help
> >>page I am unsure as to whether my dependency structure satisfies 
> the criteria
> >>to just use the regular "BH" adjustment or if  the "BY" adjustment would be
> >>more appropriate?
> >>
> >>Any suggestions for me please?
> >>
> >>Many thanks
> >>
> >>Rebecca Walls
> >>
> > Naomi S. Altman                                814-865-3791 (voice)
> > Associate Professor
> > Dept. of Statistics                              814-863-7114 (fax)
> > Penn State University                         814-865-1348 (Statistics)
> > University Park, PA 16802-2111

Naomi S. Altman                                814-865-3791 (voice)
Associate Professor
Dept. of Statistics                              814-863-7114 (fax)
Penn State University                         814-865-1348 (Statistics)
University Park, PA 16802-2111