[R] Power analysis for MANOVA?

[Stephan Kolassa]

Hi Adam,

first: I really don't know much about MANOVA, so I sadly can't help you 
without learning about it an Pillai's V... which I would be glad to do, 
but I really don't have the time right now. Sorry!

Second: you seem to be doing a kind of "post-hoc power analysis", "my 
result isn't significant, perhaps that's due to low power? Let's look at 
the power of my experiment!" My impression is that "post-hoc power 
analysis" and its interpretation is, shall we say, not entirely accepted 
within the statistical community, see:

Hoenig, J. M., & Heisey, D. M. (2001, February). The abuse of power: The 
pervasive fallacy of power calculations for data analysis. The American 
Statistician, 55 (1), 1-6

And this:
http://staff.pubhealth.ku.dk/~bxc/SDC-courses/power.pdf

However, I am sure that lots of people can discuss this more competently 
than me...

Best wishes
Stephan


Adam D. I. Kramer schrieb:
> 
> On Mon, 26 Jan 2009, Stephan Kolassa wrote:
> 
>> My (and, judging from previous traffic on R-help about power analyses,
>> also some other people's) preferred approach is to simply simulate an
>> effect size you would like to detect a couple of thousand times, run your
>> proposed analysis and look how often you get significance.  In your 
>> simple
>> case, this should be quite easy.
> 
> I actually don't have much experience running monte-carlo designs like
> this...so while I'd certainly prefer a bootstrapping method like this one,
> simulating the effect size given my constraints isn't something I've done
> before.
> 
> The MANOVA procedure takes 5 dependent variables, and determines what
> combination of the variables best discriminates the two levels of my
> independent variable...then the discrimination rate is represented in the
> statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 
> 0.71).  So
> coming up with a set of constraints that would produce V=.00019 given my
> data set doesn't quite sound trivial...so I'll go for the "par" library
> reference mentioned earlier before I try this.  That said, if anyone can
> refer me to a tool that will help me out (or an instruction manual for 
> RNG),
> I'd also be much obliged.
> 
> Many thanks,
> Adam
> 
> 
>>
>> HTH,
>> Stephan
>>
>>
>> Adam D. I. Kramer schrieb:
>>> Hello,
>>>
>>>     I have searched and failed for a program or script or method to
>>> conduct a power analysis for a MANOVA. My interest is a fairly simple 
>>> case
>>> of 5 dependent variables and a single two-level categorical predictor
>>> (though the categories aren't balanced).
>>>
>>>     If anybody happens to know of a script that will do this in R, I'd
>>> love to know of it! Otherwise, I'll see about writing one myself.
>>>
>>>     What I currently see is this, from help.search("power"):
>>>
>>> stats::power.anova.test
>>>                         Power calculations for balanced one-way
>>>                         analysis of variance tests
>>> stats::power.prop.test
>>>                         Power calculations two sample test for
>>>                         proportions
>>> stats::power.t.test     Power calculations for one and two sample t
>>>                         tests
>>>
>>>     Any references on power in MANOVA would also be helpful, though of
>>> course I will do my own lit search for them myself.
>>>
>>> Cordially,
>>> Adam D. I. Kramer
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide 
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>>
>