[R] Power analysis for MANOVA?
Adam D. I. Kramer
adik at ilovebacon.org
Thu Jan 29 23:05:02 CET 2009
Thanks for the response, Stephan.
Really, I am trying to say, "My result is insignificant, my effect sizes are
tiny, you may want to consider the possibility that there really are no
meaningful differences." Computing post-hoc power makes a bit stronger of a
claim in this setting.
My real goal in this case was to put a single line on a poster that says,
"Significance using our estimates would require N observations, which is
larger than the population." I am trying to solve for N. N in this case is
a sort of effect size. In this case, it is indeed a simple transformation
of Pillai's V and the p-value for the study, and I do not intend to suggest
that it is anything more than that. However, I believe that the latter
effect size makes a much more compelling case, given that a lot of people
(such as yourself) don't have much experience with Pillai's V.
--Adam
On Wed, 28 Jan 2009, Stephan Kolassa wrote:
> Hi Adam,
>
> first: I really don't know much about MANOVA, so I sadly can't help you
> without learning about it an Pillai's V... which I would be glad to do, but I
> really don't have the time right now. Sorry!
>
> Second: you seem to be doing a kind of "post-hoc power analysis", "my result
> isn't significant, perhaps that's due to low power? Let's look at the power
> of my experiment!" My impression is that "post-hoc power analysis" and its
> interpretation is, shall we say, not entirely accepted within the statistical
> community, see:
>
> Hoenig, J. M., & Heisey, D. M. (2001, February). The abuse of power: The
> pervasive fallacy of power calculations for data analysis. The American
> Statistician, 55 (1), 1-6
>
> And this:
> http://staff.pubhealth.ku.dk/~bxc/SDC-courses/power.pdf
>
> However, I am sure that lots of people can discuss this more competently than
> me...
>
> Best wishes
> Stephan
>
>
> Adam D. I. Kramer schrieb:
>>
>> On Mon, 26 Jan 2009, Stephan Kolassa wrote:
>>
>>> My (and, judging from previous traffic on R-help about power analyses,
>>> also some other people's) preferred approach is to simply simulate an
>>> effect size you would like to detect a couple of thousand times, run your
>>> proposed analysis and look how often you get significance. In your simple
>>> case, this should be quite easy.
>>
>> I actually don't have much experience running monte-carlo designs like
>> this...so while I'd certainly prefer a bootstrapping method like this one,
>> simulating the effect size given my constraints isn't something I've done
>> before.
>>
>> The MANOVA procedure takes 5 dependent variables, and determines what
>> combination of the variables best discriminates the two levels of my
>> independent variable...then the discrimination rate is represented in the
>> statistic (Pillai's V=.00019), which is then tested (F[5,18653] = 0.71).
>> So
>> coming up with a set of constraints that would produce V=.00019 given my
>> data set doesn't quite sound trivial...so I'll go for the "par" library
>> reference mentioned earlier before I try this. That said, if anyone can
>> refer me to a tool that will help me out (or an instruction manual for
>> RNG),
>> I'd also be much obliged.
>>
>> Many thanks,
>> Adam
>>
>>
>>>
>>> HTH,
>>> Stephan
>>>
>>>
>>> Adam D. I. Kramer schrieb:
>>>> Hello,
>>>>
>>>> I have searched and failed for a program or script or method to
>>>> conduct a power analysis for a MANOVA. My interest is a fairly simple
>>>> case
>>>> of 5 dependent variables and a single two-level categorical predictor
>>>> (though the categories aren't balanced).
>>>>
>>>> If anybody happens to know of a script that will do this in R, I'd
>>>> love to know of it! Otherwise, I'll see about writing one myself.
>>>>
>>>> What I currently see is this, from help.search("power"):
>>>>
>>>> stats::power.anova.test
>>>> Power calculations for balanced one-way
>>>> analysis of variance tests
>>>> stats::power.prop.test
>>>> Power calculations two sample test for
>>>> proportions
>>>> stats::power.t.test Power calculations for one and two sample t
>>>> tests
>>>>
>>>> Any references on power in MANOVA would also be helpful, though of
>>>> course I will do my own lit search for them myself.
>>>>
>>>> Cordially,
>>>> Adam D. I. Kramer
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide
>>>> http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>
>>>
>>
>
>
More information about the R-help
mailing list