[BioC] Multiple testing correction on 2-Way ANOVA

[Eric]

Hi Andy,

Thanks for the reply. My reasoning here is a little Byzantine so bear with me.

If the significant results are relatively evenly distributed across the 
main effects and the interaction (about the same number of genes found in 
each), then using the omnibus test will not make much of a difference. 
However, say one of the two main effects is much stronger than the other, 
then I have a case where the overall test will pick up all of those changes 
from the 'powerful' treatment (or most of them). Because of that, multiple 
testing correction at the overall level will allow genes with larger 
p-values from the second main effect through the filter compared to the 
list of genes that would make it through a multiple testing correction 
applied at the level of the second main effect.

Contrast this with the case where multiple testing is applied separately to 
each of the three outputs. Here the first main effect is relatively 
unaffected, but the second main effect is nuked (if the second main effect 
has no more genes than would be expected by chance). IMHO it doesn't matter 
what the original question was, the two multiple testing corrections change 
the list of genes and the experimental question does not address which of 
these procedures should be used. It would be disingenuous to say "Well, 
we're mainly interested in main effect 2 (the weak one), so we'll use the 
overall correction and at least see a list of genes" or "We wanted to 
disagree with previous work about main effect two's importance to research 
so we used individual correction to show the world that main effect two is 
not doing anything". Perhaps the proportion of genes assigned an 
interaction significance could be used to gauge the dependence of the two 
main effects; the more dependent they are, the more applicable the overall 
testing correction. While the smaller the proportion of genes showing an 
interaction term, the more appropriate independent correction for each main 
effect would be.


At 03:05 PM 7/27/2004, you wrote:
>I am absolutely no expert in multiple comparison / multiple testing / gene
>expression data analysis, so take the following with appropriate dose of
>salt:
>
>It really depends on what you are looking to get out of the data.  Just
>because you have multi-factor data with > 2 levels and thousands of
>responses, it doesn't automatically mean that the usual multiple comparison
>procedures are appropriate.  You design the experiment to answer some
>specific questions (hopefully).  How you analyze the data depends greatly on
>what those questions are, and (hopefully, therefore) how the experiment is
>designed.
>
>Best,
>Andy
>
> > From: Eric
> >
> > Hi,
> >
> > I apologize for this being off-topic- it's really a
> > statistical question
> > but I'd be interested in the community's input. If I run a
> > 'per gene' 2-way
> > ANOVA on single channel microarray data (i.e., each gene is tested
> > separately by 2-Way ANOVA), should I run multiple testing
> > correction for
> > each factor and interaction separately? Alternatively, should
> > I use an
> > overall (omnibus) F-test, correct that for multiple testing,
> > and treat the
> > main effects and interaction results as post-hoc to the overall test?
> >
> > Thanks,
> > -E
> >
> > Eric Blalock, PhD
> > Dept Pharmacology, UKMC
> > 859 323-8033
> >
> > STATEMENT OF CONFIDENTIALITY\ \ The contents of this e-mail
> > ...{{dropped}}
> >
> > _______________________________________________
> > Bioconductor mailing list
> > Bioconductor at stat.math.ethz.ch
> > https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor
> >
> >
>
>
>------------------------------------------------------------------------------
>Notice:  This e-mail message, together with any attachments, contains 
>information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station, New 
>Jersey, USA 08889), and/or its affiliates (which may be known outside the 
>United States as Merck Frosst, Merck Sharp & Dohme or MSD and in Japan, as 
>Banyu) that may be confidential, proprietary copyrighted and/or legally 
>privileged. It is intended solely for the use of the individual or entity 
>named on this message.  If you are not the intended recipient, and have 
>received this message in error, please notify us immediately by reply 
>e-mail and then delete it from your system.
>------------------------------------------------------------------------------

Eric Blalock, PhD
Dept Pharmacology, UKMC
859 323-8033

STATEMENT OF CONFIDENTIALITY\ \ The contents of this e-mail ...{{dropped}}