[R] explaining curious result of aov

[(Ted Harding)]

On 21-Oct-03 Bill Shipley wrote:
> Hello.  I have come across a curious result that I cannot explain.
> Hopefully, someone can explain this.  I am doing a 1-way ANOVA with 6
> groups (example: summary(aov(y~A)) with A having 6 levels).  I get an F
> of 0.899 with 5 and 15 df (p=0.51).  I then do the same analysis but
> using data only corresponding to groups 5 and 6.  This is, of course,
> equivalent to a t-test.  I now get an F of 142.3 with 1 and 3 degrees
> of freedom and a null probability of 0.001.  I know that multiple
> comparisons changes the model-wise error rate, but even if I did all 15
> comparisons of the 6 groups, the Bonferroni correction to a 5% alpha is
> 0.003, yet the Bonferroni correction gives conservative rejection
> levels.
> 
> How can such a result occur?  Any clues would be helpful.

It's not obvious from your description. However, one possibility (which
I very strongly suspect) is apparent heterogeneity of variance, coupled
with paucity of data.

To wit: The denominator in F is the residual sum of squares (divided by
its degrees of freedom -- 15 in your first case, 3 in your second).

If the data in groups 5 and 6 are very close to their group means,
the group means themselves being more widely separated, then you can
indeed get a large F. The very moderate F that you get from the full
set of groups is quite compatible with the extreme result from the
two-group analysis if the data happen to be more widely spread about
their group means than they happen to be in G5+G6. This is the
"heterogeneity of variance" side of it.

Your denominator df = 3 for the two-group case indicates that you
only have 5 data values altogether in these two groups. Your df = 15
for the six-group case indicates that you have only 21 data all told.
At an average of 3.5 data per group you have a very thin data set.
Your 2.5 data per group in G5+G6 is even thinner. I would be very
cautious about interpreting the results in such a case.

Perhaps if you told us more about your data we could give a more
focussed diagnosis.

Best wishes,
Ted.


--------------------------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding at nessie.mcc.ac.uk>
Fax-to-email: +44 (0)870 167 1972
Date: 21-Oct-03                                       Time: 17:56:53
------------------------------ XFMail ------------------------------