[R] Difference in ANOVA results - R vs. JMP/Minitab

[John Fox]

Dear Nirmal,

At 07:36 AM 11/19/2003 -0500, Nirmal Govind wrote:

>I ran a small data set from a factorial experiment through R, Minitab and 
>JMP... the result from R is significantly different from what Minitab or 
>JMP give... The data set is at the following link:
>
>http://www.personal.psu.edu/nug107/Uploads/2x3_16repsANOVA.txt
>
>The first 5 columns are the factors and the next three are responses.
>In particular, for the response beta11MSE, two of the significant effects 
>in the Minitab (and JMP) ANOVA table are not significant in R...  What may 
>be the cause for such a difference? I used the lm function in R to do the 
>model fitting and fit a model with main effects and 2-way interactions in 
>the 5 factors. The design that generated this data was a 32-run 2^5 full 
>factorial.

 From your description, it's not possible to tell what you did or exactly 
what happened -- either in R or in Minitab (and JMP).

Moreover, bcause the design is orthogonal and the factors are dichotomous, 
it's very difficult to get the analysis wrong, at least in R. I believe 
that you should get correct sums of squares even if you don't treat the 
factors as factors, regardless of contrast coding, and even if you use 
"sequential" SSs, as calculated by the anova function (all of which could 
get you in trouble more generally).

Here are the results that I get from R and SAS (I don't have Minitab or JMP):

 > anova(lm(beta11MSE ~ (Cycles + Replns + VarZ + NormB + NormG)^2))
Analysis of Variance Table

Response: beta11MSE
               Df  Sum Sq Mean Sq F value    Pr(>F)
Cycles         1 0.00102 0.00102  0.1602  0.694250
Replns         1 0.18457 0.18457 28.9170 6.164e-05 ***
VarZ           1 0.26800 0.26800 41.9879 7.597e-06 ***
NormB          1 0.00491 0.00491  0.7689  0.393527
NormG          1 0.33606 0.33606 52.6497 1.921e-06 ***
Cycles:Replns  1 0.00004 0.00004  0.0059  0.939751
Cycles:VarZ    1 0.00131 0.00131  0.2060  0.656054
Cycles:NormB   1 0.00008 0.00008  0.0131  0.910348
Cycles:NormG   1 0.00088 0.00088  0.1377  0.715457
Replns:VarZ    1 0.07781 0.07781 12.1897  0.003018 **
Replns:NormB   1 0.00360 0.00360  0.5636  0.463697
Replns:NormG   1 0.08693 0.08693 13.6199  0.001982 **
VarZ:NormB     1 0.00351 0.00351  0.5498  0.469155
VarZ:NormG     1 0.08990 0.08990 14.0843  0.001737 **
NormB:NormG    1 0.00268 0.00268  0.4206  0.525839
Residuals     16 0.10213 0.00638


proc glm;
   class Cycles Replns VarZ NormB NormG;
   model beta11MSE = Cycles|Replns Cycles|VarZ Cycles|NormB Cycles|NormG
   Replns|VarZ Replns|NormB Replns|NormG VarZ|NormB VarZ|NormG NormB|NormG;
   run;

       Source                      DF       Type I SS     Mean Square    F 
Value    Pr > F

       Cycles                       1      0.00102265      0.00102265 
0.16    0.6942
       Replns                       1      0.18457369      0.18457369 
28.92    <.0001
       Cycles*Replns                1      0.00003763      0.00003763 
0.01    0.9398
       VarZ                         1      0.26800351      0.26800351 
41.99    <.0001
       Cycles*VarZ                  1      0.00131456      0.00131456 
0.21    0.6561
       NormB                        1      0.00490793      0.00490793 
0.77    0.3935
       Cycles*NormB                 1      0.00008353      0.00008353 
0.01    0.9103
       NormG                        1      0.33605652      0.33605652 
52.65    <.0001
       Cycles*NormG                 1      0.00087885      0.00087885 
0.14    0.7155
       Replns*VarZ                  1      0.07780526      0.07780526 
12.19    0.0030
       Replns*NormB                 1      0.00359764      0.00359764 
0.56    0.4637
       Replns*NormG                 1      0.08693408      0.08693408 
13.62    0.0020
       VarZ*NormB                   1      0.00350913      0.00350913 
0.55    0.4692
       VarZ*NormG                   1      0.08989860      0.08989860 
14.08    0.0017
       NormB*NormG                  1      0.00268461      0.00268461 
0.42    0.5258


       Source                      DF     Type III SS     Mean Square    F 
Value    Pr > F

       Cycles                       1      0.00102265      0.00102265 
0.16    0.6942
       Replns                       1      0.18457369      0.18457369 
28.92    <.0001
       Cycles*Replns                1      0.00003763      0.00003763 
0.01    0.9398
       VarZ                         1      0.26800351      0.26800351 
41.99    <.0001

       . . .


So, what's the problem?

I hope that this helps,
  John
-----------------------------------------------------
John Fox
Department of Sociology
McMaster University
Hamilton, Ontario, Canada L8S 4M4
email: jfox at mcmaster.ca
phone: 905-525-9140x23604
web: www.socsci.mcmaster.ca/jfox