[R] exact wilcox test and Bonferroni correction

[Meyners, Michael, LAUSANNE, AppliedMathematics]

(i) you EITHER correct the p value (by multiplying by 8 in your case) OR
you use the Bonferroni-threshold of 0.05/8, not both. If you correct the
p values, your threshold remains 0.05. If you use 0.05/8, you use the
original p values.
(ii) Yes, if the p value is 0.15, then the corrected one for 8 tests is
min(0.15*8, 1)=1 (i.e. all p values of 1/8 and higher will be 1 after
correction)
(iii) No, you can NOT AT ALL conclude that your samples are from the
same distribution. You have no evidence that they are from different
ones, but that does not imply that they come from the same (or a
similar) one. In particular as the power of the test can be assumed to
be pretty low.

Regarding (i), it might be helpful to search the web for some
information on Bonferroni correction, you could start with wikipedia,
but there are other resources around. Regarding (iii), I would strongly
recommend (re-)reading a good introduction into stats or consulting an
expert statistician. I'm afraid you are missing some of the basics of
statistical hypothesis testing here. To conclude "similarity", you'd
have to define first what exactly that means (you can NEVER show exact
equality), and then construct some appropriate test for that. 

NB: I am sure that you won't succeed to show similarity with a
reasonable definition if you have just 10 observations, though. And for
completeness: If you were aiming at similarity, you should rather NOT
correct for multiplicity (while that depends again on what exactly you
want to show). But I think this goes far beyond the scope of this post
now. 

HTH, Michael
 

> -----Original Message-----
> From: r-help-bounces at r-project.org 
> [mailto:r-help-bounces at r-project.org] On Behalf Of netrunner
> Sent: Freitag, 22. Januar 2010 15:25
> To: r-help at r-project.org
> Subject: Re: [R] exact wilcox test and Bonferroni correction
> 
> 
> Dear Michael,
> thank you very much for your help.
> 
> I perfomed the wilcox.exact function on each of the 8 items 
> for the two groups that I am analysing (that is, I performed 
> 8 times the wilcox test).
> Here an example for the values (ratings from a questionnaire) 
> of one of the
> 8 items:
> 
> a1=5  9 10 10 10 10 10 10 10 10
> 
> b1=7   5  10 NaN  10  10   8  10  10   8   9   9
> 
> wilcox.exact(a1,b1, alternative="two.sided", mu=0, 
> paired=FALSE, exact=TRUE,
> conf.level=0.95)
> 
> I obtained:
> 
> data:  a1 and b1
> W = 73.5, p-value = 0.1514
> alternative hypothesis: true mu is not equal to 0 
> 
> Then I adjusted p-values using p.adjust. 
> 
> For the example above the p-bonferroni value was 1. The 
> threshold p-value is
> 0.00625 (that is 0.05/8)
> Finally, because p-bonferroni > 0.00625 can I conclude that 
> for each item my samples are from the same distribution?
> 
> I am a little bit confused....
> 
> thank you!
> 
> netrunner
> 
> 
> --
> View this message in context: 
> http://n4.nabble.com/exact-wilcox-test-and-Bonferroni-correcti
on-tp1099893p1100077.html
> Sent from the R help mailing list archive at Nabble.com.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>