[R] power analysis is applicable or not

[David Winsemius]

On Nov 12, 2013, at 6:10 PM, array chip wrote:

> Hi, this is a statistical question rather than a pure R question. I have got many help from R mailing list in the past, so would like to try here and appreciate any input:
> 
> I conducted Mantel-Haenszel test to show that the performance of a diagnostic test did not show heterogeneity among 4 study sites, i.e. Mantel Haenszel test p value > 0.05,  so that I could conduct a meta-analysis by combining data of all 4 study sites. 
> 
> Now one of the reviewers for the manuscript did a powering analysis for Mantel Haneszel test showing that with the sample sizes I have, the power for Mantel Haeszel test was only 50%. So he argued that I did not have enough power for Mantel Haenszel test.
> 
> My usage of Mantel Haenszel was NOT to show a significant p value, instead a non-sginificant p value was what I was looking for because non-significant p value indicate NO heterogeneity among study sites. Powering analysis in general is to show whether you have enough sample size to ensure a statistical significant difference can be seen with certain likelihood. But this is not how I used Mantel Haenszel test. So I think in my scenario, the power analysis is NOT applicable because I am simply using the test to demonstrate a non-significant p value.
> 
> Am I correct on this view?

I think not. If you need a p > 0.05 to argue for lumping categories together then you should have sufficient power for the test. Now since you havetoldus almost nothing about the science or the analysis, this is a very general opinion that could get modified if there were further specifics presented. In particular it is often the case that MH tests are done on ordered categories but the ordinary MH tests fail to account for that aspect.

-- 

David Winsemius
Alameda, CA, USA