[BioC] questions on results from mt.maxT (multtest)

[Wolfgang Huber]

Hi Perry,

in fact Wilcoxon-test will suffer from similar granularity as a 
permutation test. A straight-forward t-test is also problematic, due to 
the small number of replicates and the resulting high variance of the 
denominator in the t-test. My advice would be to use a moderated t-test 
such as available through the eBayes function in limma, well-described 
in the vignette, or in fact, the user's guide.

  Best wishes
   Wolfgang


P.D. Moerland wrote:
> Hi Juerg,
> 
> what Wolfgang means is that in your case with six samples, three from
> each class, you can only have (6 choose 3)= 20 permutations of the class
> labels. That's why your p-values will always be multiples of 1/20. A
> permutation test is not suitable with so few samples; you should use one
> of the standard tests (t, Wilcoxon, limma's robust t) and then correct
> for mulitple testing.
> 
> Best wishes,
> Perry
> 
> ----- Original Message -----
> From: "Straubhaar, Juerg" <Juerg.Straubhaar at umassmed.edu>
> Date: Monday, February 14, 2005 5:08 pm
> Subject: RE: [BioC] questions on results from mt.maxT (multtest)
> 
> 
>>Hi Wolfgang,
>>
>>I don't quite understand what you mean by 'parametric test': the 
>>mt.maxT function has a parameter 'nonpara' which is set by default 
>>to 'n'. I assume a parametric test was performed.
>>
>>Juerg
>>
>>
>>-----Original Message-----
>>From: Wolfgang Huber [mailto:huber at ebi.ac.uk]
>>Sent: Sun 2/13/2005 12:12 PM
>>To: Straubhaar, Juerg
>>Cc: bioconductor at stat.math.ethz.ch
>>Subject: Re: [BioC] questions on results from mt.maxT (multtest)
>>
>>Hi Juerg,
>>
>>the p-values from a permutation test are multiples of 1/(number of 
>>permutations). If you want p-values with more digits, you need to 
>>do a 
>>parametric test.
>>
>>You say your exprs matrix has 6 colums, but in the call to mt.maxT 
>>you 
>>only have 5 classlabels. Wrong?
>>
>> Bw
>>  Wolfgang
>>
>>
>>Straubhaar, Juerg wrote:
>>
>>>Im am using R version 2.0.1, multtest version 1.5.2.
>>>
>>>Microarray experiment: wildtype tissue vs KO tissue; affy MOE430_2
>>>Experiments in triplicates.
>>>
>>>ExprSet at exprs matrix has dimension: 45101 6
>>>
>>>I used nonspecific filtering which reduces the number of genes to 
>>
>>1815 (exprSub)
>>
>>>I am using multtest to get a set of differentially expressed genes:
>>>
>>>resT <-mt.maxT(exprSub at exprs, c(0,1,0,1,0))
>>>This runs 20 permutations (complete enumeration)
>>>
>>>sort(resT$rawp) looks like this:
>>>
>>>[1] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 
>>
>>0.1 0.1 0.1 0.1
>>
>>>[20] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 
>>
>>0.1 0.1 0.1 0.1
>>
>>>[39] 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
>>>...
>>>[1711] 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 
>>
>>0.9 0.9 0.9 0.9
>>
>>>[1729] 0.9 0.9 0.9 0.9 0.9 0.9 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 
>>
>>1.0 1.0 1.0 1.0
>>
>>>[1747] 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 
>>
>>1.0 1.0 1.0 1.0
>>
>>>[1765] 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 
>>
>>1.0 1.0 1.0 1.0
>>
>>>[1783] 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 
>>
>>1.0 1.0 1.0 1.0
>>
>>>[1801] 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
>>>
>>>It looks like these p-values are rounded to one decimal digit. 
>>
>>From these results none of the 1815  genes are differentially 
>>expressed between WT and KO, which is not quite believable. I would 
>>like to get p-values precise to several digits. Maybe there is 
>>something else I don't understand.
>>
>>>getOption('digits') 7
>>>
>>>Thank you.
>>>
>>>Juerg Straubhaar
>>>Umass Med
>>>
>>>_______________________________________________
>>>Bioconductor mailing list
>>>Bioconductor at stat.math.ethz.ch
>>>https://stat.ethz.ch/mailman/listinfo/bioconductor
>>
>>
>>-- 
>>Best regards
>>  Wolfgang
>>
>>-------------------------------------
>>Wolfgang Huber
>>European Bioinformatics Institute
>>European Molecular Biology Laboratory
>>Cambridge CB10 1SD
>>England
>>Phone: +44 1223 494642
>>Fax:   +44 1223 494486
>>Http:  www.ebi.ac.uk/huber
>>
>>_______________________________________________
>>Bioconductor mailing list
>>Bioconductor at stat.math.ethz.ch
>>https://stat.ethz.ch/mailman/listinfo/bioconductor
>>
> 
> 
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/bioconductor


-- 
Best regards
   Wolfgang

-------------------------------------
Wolfgang Huber
European Bioinformatics Institute
European Molecular Biology Laboratory
Cambridge CB10 1SD
England
Phone: +44 1223 494642
Fax:   +44 1223 494486
Http:  www.ebi.ac.uk/huber