[BioC] Why NONE of any contrast is significant when overall F-test is significant? using "global" method in decideTests() from limma package

[James W. MacDonald]

Hi Guiyan,

Guiyuan Lei wrote:
> Dear all,
> 
> I am using limma package to identify differential expression. I have 5
> contrasts, I used F-statistic to measure significant differential
> expression,  the F-test p-value is adjusted by  "fdr" method . I used
> the command
> 
> p.adjust(eb$F.p.value, method="fdr") < 0.05
> 
> where eb is the object from eBayes(). I got 397 significant
> differential expression.
> 
> Then for those 397 significantly differential expressed genes, I want
> to look at each contrast to check which contrast is significant. I
> used
> 
> decideTests(eb, method="global")
> 
> to classify each t-statistics as up, down or not significant.
> 
> I found that among genes which are significantly expressed by F-test
> p-value, that is, the above 397 genes, some have got five '0's from
> decideTests, which means those genes are not significant for any of
> its contrast while its F-test is significant. As I understand, if
> F-test is significant, should at least one of its contrast (t-test) is
> significant. I doubt that the "global" method used in decideTests() is
> not properly used for this case? But why? Can any one explain this to
> me? Many thanks! I attach the code and results as following.

Yes. From ?decideTests:

'method="global"' will treat the entire matrix of t-statistics as
      a single vector of unrelated tests.

So when you did the F-test and adjusted for multiplicity, you adjusted 
for n tests. Then when you did the t-tests, you did the same, but 
adjusting for 5n tests. Increasing the number of simultaneous tests 
five-fold was enough to cause your multiplicity-adjusted p-values to all 
become insignificant at an alpha of 0.05.

You might try method="separate" or "nestedF".

Best,

Jim




> 
> library(limma)
> levels= c('h','h','h','hh','hh','mh','mh','s','s','s','y','y','y','y')
> dimnames(eset.matrix)[[2]]= levels
> TS <- factor(levels, levels= c('h','hh','mh','s','y'))
> design <- model.matrix(~0+TS)
> colnames(design) <- levels(TS)
> fit <- lmFit(eset.gcrma, design)
> 
> #Construct the contrasts
> mc <- makeContrasts('s-y','h-y','mh-y','hh-h','mh-hh',levels=design)
> fit2 <- contrasts.fit(fit, mc)
> eb <- eBayes(fit2)
> 
> #The adjustment methods using "fdr"
> modFpvalue <- eb$F.p.value
> selectedgenesindx <- p.adjust(eb$F.p.value, method="fdr") < 0.05
> Sig<-modFpvalue[selectedgenesindx]
> nsiggenes<-length(Sig) #number of significantly differential expression
> 
> #decideTests using "global" method
> results1 <- decideTests(eb, method="global")
> 
> #Order the F-statistic
> modF <- eb$F
> modFordered<-order(modF, decreasing = TRUE)
> 
> #Get the up or down or not significant for nsiggenes significant
> (according adjusted p-value of F-test) differential expression
> updown<-results1[modFordered[1:nsiggenes],]
> 
> Examples for genes which are significant expression by F-test
> F-test           results from decideTests()
> eb$F.p.value  contrast1  contrast2 contrast3  contrast4  contrast4
>  5.6E-05 	 0	       0	           0	         0             0
> (this one is not significant for any of its contrast, why five '0's?)
> 0.00036          0	      -1	           0	         0	       0									
> 
> 
> Best regards,
> Guiyuan
> 
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/bioconductor
> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor

-- 
James W. MacDonald, M.S.
Biostatistician
Affymetrix and cDNA Microarray Core
University of Michigan Cancer Center
1500 E. Medical Center Drive
7410 CCGC
Ann Arbor MI 48109
734-647-5623