[R] Multiply Iterated Measurements and Pairwise Comparison

[Leif Kirschenbaum]

I am familiar with pairwise t-tests, corrections for multiple testing, etc. however I have a problem whose answer I have not found after extensive R-help archive and Google searching.

What I have done in the past:
I have N items which are measured, exposed to a condition, and then measured again.  I wish to know if the condition changes the items so I can perform a t-test.  Better yet I can perform a pairwise t-test because the items are individually identifiable.
Sometimes the N items measures do not appear normal and I have used tests other than the t-test (chi-square, etc.)
Sometimes the N items are measured after each one of several exposures and I used pairwise.t.test().


I have a situation where N is limited and I cannot increase it, therefore I arranged for M measurements of the N items to be made before and after exposure to the condition.  Thus I have N x M measurements before the condition and N x M measurements after the condition.  I assumed that performing M measures instead of 1 measure I would be able to provide more statistical power, however I don't know what method to use.

I could:
(1) average the M measurements to obtain just N measures for the N items and perform pairwise t-testing.
This seems to lose statistical weight.

(2) make believe that I actually have N x M items and perform pairwise t-testing.
This will give me a more powerful result than (1), however I feel like I am losing something by saying I have N x M independent items instead of M measures a piece of N items.

Any suggestions how to statistically test whether the condition changed the N items?
I am looking for a p-value to indicate whether there was a significant change or not.

P.S. Yes, I actually have more than one condition, so the N items are measured M times, exposed to condition 1, measured M times, exposed to condition 2, ... and I plan to set a stringent enough confidence level to avoid Bonferroni problems.  I have also tried pairwise.t.test() with the Holm method for p-value adjustment, however I don't see that pairwise.t.test() has a functionality to accommodate my M measurements arrangement.

Thank you for any suggestions.


Leif S. Kirschenbaum, Ph.D., PMP, CRE
Design Reliability
Product Reliability
Space Systems/Loral
3825 Fabian Way M/S H-21
Palo Alto, CA 94303
650-852-6580