[R] Power of t-test in R vs. S-PLUS
Thomas Lumley
tlumley at u.washington.edu
Fri Mar 1 23:29:25 CET 2002
On Fri, 1 Mar 2002, Alan T. Arnholt wrote:
> Could someone shed some light on the statement from Thomas
> that "(with sampling uncertainties of about +/- 2% in each
> number)". What exactly is being said about the accuracy of
> the simulation and how is that number +/-2% being
> determined.
>
The power is estimated by doing 10000 simulated experiments and finding
that 7977 of them give a p-value below 0.01.
The number of p-values less than 0.01 is random (these are simulations),
and has a Binomial(10000, P) distribution, where P is the power. It is
possible to construct an exact 95% confidence interval for P based on the
observed results of the simulations. The binom.test function does this and
says that a 95% ci for the power is (0.790,0.806). Similarly for the
delta specified by S-PLUS the 95% ci for the power is (0.537,0.557). The
sampling error is actually about +/- 1% (I was incorrectly rounding in my
previous email).
If you had good parameter values to use in a power calculation and wanted
to be as precise in your claims as possible you might do a simulation like
this and quote a suitable lower confidence limit for the power. You can
get very tight limits if you want. For example, a lower 99.99% confidence
limit for the power based on this simulation is 0.782.
It matters more when the problem is more complicated and you can't readily
do 10000 replicates in the simulation.
-thomas
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
More information about the R-help
mailing list