[R] observed power

[Marc R. Feldesman]

An article relevant to this discussion can be found in:

Hoenig JM and Heisey DM. 2001.  The abuse of power:  the pervasive fallacy 
of power calculations for data analysis.  American Statistician 55(1): 19 - 
24.   (This is the Feb 2001 issue, which I just received yesterday).

Abstract:

It is well known that statistical power calculations can be valuable in 
planning an experiment.  There is also a large literature advocating that 
power calculations be made whenever one performs a statistical test of a 
hypothesis and when one obtains a statistically non-significant 
result.  Advocates of such post-experiment power calculations claim the 
calculations should be used to aid in the interpretation of the 
experimental result.  This approach, which appears in various forms, is 
fundamentally flawed.  We document that the problem is extensive and 
present arguments to demonstrate the flaw in the logic.


There is a whole section on "Observed Power" listed under "Inappropriate 
uses of power analysis".







At 12:07 PM 1/27/01 +1000, Bill.Venables at CMIS.CSIRO.AU wrote:
 >
 >
 >Peter Dalgaard BSA [mailto:p.dalgaard at biostat.ku.dk] wonders:
 >
 >| Sent: Saturday, 27 January 2001 1:29
 >| To: Mark M. Span
 >| Cc: r-help at stat.math.ethz.ch
 >| Subject: Re: [R] observed power
 >|
 >|
 >| "Mark M. Span" <span at psy.uva.nl> writes:
 >|
 >| > Is there a way to obtain the observed power of an aov()?
 >| >
 >| > I perform an aov with one between and one within factor,
 >| > and would like to know the observed power of the tests,
 >| > both for the main effect and the interaction. I found the
 >| > package 'hpower', but sense there is a more convenient
 >| > possibility. Is there?
 >| >
 >| > thanks
 >| >
 >| > Mark M. Span
 >|
 >| What's "observed power"? If you mean the item that SPSS has by that
 >| name, I think you first have to convince us that that is a sensible
 >| thing to calculate...
 >
 >If you ever do find out, Peter, let me know too, please.  I was puzzled, but
 >a bit worried about showing my ignorance...   I can only imagine it means an
 >estimate of the non-centrality parameter in which case it is a sensible
 >thing to have available since it is essentially the signal-to-noise ratio.
 >Actually the MLE is the F-statistic but the maximum *marginal* likelihood
 >estimate (based on the marginal distribution of the F-statistic itself) is
 >of more interest as it is closer to unbiased.  In the case of the multiple
 >correlation coefficient, for example, this is (practically) what people call
 >the "adjusted R^2" statistic, where the adjustment is essentially a bias
 >correction.  You can come up with simple analogues for non-central
 >chi-squared and non-central F of course, but they are again just simple
 >linear adjustments, unless you really want to get flash.  (I wrote a couple
 >of papers on this stuff in the 70s so I have a kind of nostalgic affinity
 >for it...)
 >
 >I would be more interested in these quantities optionally appearing
 >routinely on summary tables than, for example, the cute 'significance
 >stars'.  But as for calling them the "observed power", I would definitely
 >caution against that.  It encourages entirely the wrong idea of what power
 >really is.  (For example, it is a function, not a quantity, and you don't
 >ever "observe" it in practice.)
 >
 >Bill Venables.
 >--
 >Bill Venables, CSIRO/CMIS Environmetrics Project
 >Email: Bill.Venables at cmis.csiro.au
 >Phone: +61 7 3826 7251
 >Fax:   +61 7 3826 7304
 >Postal: PO Box 120, Cleveland, Qld 4163, AUSTRALIA
 >
 >-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
 >r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
 >Send "info", "help", or "[un]subscribe"
 >(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
 >_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._


Dr. Marc R. Feldesman
email:  feldesmanm at pdx.edu
email:  feldesman at attglobal.net
fax:    503-725-3905

"Don't know where I'm going.
Don't like where I've been.
There may be no exit.
But hell, I'm going in."  Jimmy Buffett

Powered by Superchoerus - the 700 MHz Coppermine Box

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._