[BioC] Minimal group sizes in permutation tests
James W. MacDonald
jmacdon at med.umich.edu
Tue Nov 13 15:04:26 CET 2007
Benjamin Otto wrote:
> Hey all,
> 1. For statistical tests there are usually minimal group sizes recommended
> for appropriate working. For a chi-square test as example the lower level
> was 10 obersvations in each field of the table, if I remember correctly.
> What about permutation tests? Is there some kind of minimal recommendation
> for group sizes? I can't find any hint on that.
The lowest p-value you can get (aside from 0) will be 1/n, where n == #
of combinations. The usual recommendation for any sort of
permutation/sampling test (bootstrap, jackknife, etc) is 500 - 1000
permutations, so you need at least 6 samples per group if you want to
follow that recommendation.
> 2. As far as I understand the permutation p-value is given by the quantile
> describing the position of the native p-value in the permutation p-value
> distribution. So for 100 permutations and 5 values smaller than the native
> one the new p-value would be 0.05. What happens when the original p-value is
> the absolut minimum? Is such a thing like p-value equals zero defined?
Sure. If the observed statistic is larger than _all_ the permuted
statistics, then the permuted p-value is 0/n. Of course if your permuted
null is really coarse (say only 20 combinations) then this is less
impressive than if you had much more.
> 3. Given a design of 3x3 samples (20 permutations), will the test return
> reasonable values? Doesn't look like it to me.
Well your permuted null will be very coarse. IMO, in this case using
standard normal theory is preferable.
> Best regards,
> Benjamin Otto
> Benjamin Otto
> University Hospital Hamburg-Eppendorf
> Institute For Clinical Chemistry
> Martinistr. 52
> D-20246 Hamburg
> Tel.: +49 40 42803 1908
> Fax.: +49 40 42803 4971
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor
James W. MacDonald, M.S.
Affymetrix and cDNA Microarray Core
University of Michigan Cancer Center
1500 E. Medical Center Drive
Ann Arbor MI 48109
More information about the Bioconductor