[R] significant anova but no distinct groups ?

[Cody_Hamilton at Edwards.com]

Frederic,

You're performing 8*7/2 = 28 multiple comparisons controlling the FWE at
the .05 level.  Using a Bonferroni's adjustment (admittedly more
conservative than the Holm's or Tukey's approach), that's testing each
comparison at the .05/28 = 0.0018 level.  With only 100 observations spread
over 8 levels, you won't have much power to detect a difference.

Regards,
   -Cody



                                                                           
             Frederic Jean                                                 
             <Frederic.Jean at un                                             
             iv-brest.fr>                                               To 
             Sent by:                  r-help at r-project.org                
             r-help-bounces at st                                          cc 
             at.math.ethz.ch                                               
                                                                   Subject 
                                       [R] significant anova but no        
             03/02/2007 01:52          distinct groups ?                   
             PM                                                            
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




Dear all,

I am studying a dataset using the aov() function.

The independant variable 'cds' is a factor() with 8 levels and here is
the result in studying the dependant variable 'rta' with aov() :

> summary(aov(rta ~ cds))
             Df  Sum Sq Mean Sq F value  Pr(>F)
cds          7 0.34713 0.04959  2.3807 0.02777
Residuals   92 1.91635 0.02083

The dependant variable 'rta' is normally distributed and variances are
homogeneous.
But when studying the result with TukeyHSD, no differences in 'rta'
are seen among groups of 'cds' :

> TukeyHSD(aov(rta ~ cds), which="cds")
   Tukey multiple comparisons of means
     95% family-wise confidence level

Fit: aov(formula = rta ~ cds)

$cds
              diff        lwr        upr     p adj
1-0 -0.1046092796 -0.4331100 0.22389141 0.9751178
2-0  0.0359991860 -0.1371359 0.20913425 0.9980970
3-0  0.0261665235 -0.1348524 0.18718540 0.9996165
4-0  0.0004502442 -0.1805448 0.18144531 1.0000000
5-0 -0.1438949939 -0.3104752 0.02268526 0.1422670
[...]
7-5  0.0621598639 -0.1027595 0.22707926 0.9386170
7-6  0.0256519274 -0.1757408 0.22704465 0.9999248

I tried a pairwise.t.test (holm correction) which also was not able to
detect differences in 'rta' among groups of 'cds'
I've never been confronted to such a situation before : is it just a
problem of power of the /a posteriori/ tests used ? Do I miss
something important in basic stats or in R ?
How to highlight differences among 'cds' groups seen with aov() ?

Any help appreciated
Thanks in advance,

Fred J.

______________________________________________
R-help at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.