[R] How to compute the power of a wilcoxon test?

[peter dalgaard]

On Sep 15, 2011, at 17:15 , tn85 wrote:

> Thank you, Peter. They are empirical quantities, that's why I felt confused as being requested to give the power of my test. Could you please clarify the reason why power calculations are not fit for empirical quantities? So that I can defend my test for such request.

Well, in two words: "Too late". Power calculations belong before the data are collected, once you have them, p-values and confidence intervals provide all relevant information. You can Google for "post hoc power" and find discussions involving more than two words...

One exception could be when it is considered useful to know what size of effect the study was designed to find. If authors "forgot" to do so in advance, they might want to reconstruct the calculation later. This verges on scientific misconduct (of a relatively minor sort), but even in those cases, I'd say that the observed differences are irrelevant and you should do the calculation AS IF you were starting from scratch: decide on the minimum relevant difference, etc. 

> 
> BTW, the correlation of the two samples is -0.053.

So they really are paired? If so, the code I gave should probably have paired=TRUE in the test, but simulating as uncorrelated is defensible.

> 
> Thanks,
> Tianchan
> 
> From: Peter Dalgaard-2 [via R] [mailto:ml-node+s789695n3815852h43 at n4.nabble.com]
> Sent: Thursday, September 15, 2011 11:04 AM
> To: Tianchan Niu
> Subject: Re: How to compute the power of a wilcoxon test?
> 
> 
> On Sep 15, 2011, at 15:47 , tn85 wrote:
> 
>> Hello All,
>> 
>> I posted a similar question before, but the direction was driven to whether
>> my case is suitable for a wilcoxon test. After research about the
>> appropriateness, I am pretty sure that a wilcoxon test is the right tool for
>> my case. But how to compute the power of the test is still an unanswered
>> question bothering me.
>> 
>> The basic stats of my two paired samples are:
>> mean1 = 0.0032,   sd1 = 0.012
>> mean2 = 0.00042, sd2 = 0.0016
>> sample size = 366
>> 
>> Could anyone help? Thank you in advance.
> 
> Thing is, you can't really do that. Nonparametic tests are only distribution-free under the null hypothesis. For power, you need the distribution of the test statistic under the alternative.
> 
> First: Do you really mean "paired"? If so, there would seem to be some information missing, namely the magnitude of the correlation. Also, do you really mean that the sd's differ by a factor of 10?? (And, if those are empirical quantities, don't do power calculations at all).
> 
> Assuming that you have two independent samples of size 366, and parameters as above, you might do
> 
>> pval <- replicate(1000, wilcox.test(rnorm(366,.0032,.012), rnorm(366,.00042,.0016))$p.value)
>> summary(pval)
>     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
> 0.0000000 0.0000004 0.0000185 0.0064520 0.0003877 0.8997000
>> sum(pval < .05)
> [1] 979
> 
> suggesting a power of around 98%. Adjust as necessary.
> 
>> 
>> --
>> View this message in context: http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815616.html
>> Sent from the R help mailing list archive at Nabble.com.
>> 
>> ______________________________________________
>> [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=0> mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> 
> --
> Peter Dalgaard, Professor
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Email: [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=1>  Priv: [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=2>
> 
> ______________________________________________
> [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=3> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815852.html
> To unsubscribe from How to compute the power of a wilcoxon test?, click here<http://r.789695.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3815616&code=bml1QGlzaXMuZ2VvcmdldG93bi5lZHV8MzgxNTYxNnwtMTI4NTg5ODAwMQ==>.
> 
> 
> --
> View this message in context: http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815893.html
> Sent from the R help mailing list archive at Nabble.com.
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com