Hi,

I am performing experiments in the field of visual perception where  
we often apply balanced designs. Within a group of normal subjects,  
we vary different stimulus conditions (like contrast, luminance,  
temporal frequency of stimulation) and derive some psychophysical or  
electrophysiological results from our subjects. Often, the main  
question is to test the effect of these parameters on the recorded  
results.

When switching from Statview (old software on Mac computers, no  
longer supported) to R I learned that the problem is translated  to   
a repeated measures ANOVA via

ano <- aov(d$Result~ d$Condition1*d$Condition2 + Error(d$Subject/(d 
$Condition1*d$Condition2), data=d))

However, there are problems in performing post-hoc tests due to the  
intraindividual correlation of the repeated measures data.
So I started intense online searches for a good solution in R, found  
snippets of R-code here and there. One problem was that many  
contributions offered help for the first step to replace the "aov"  
procedure with a call to "lme" in the nlme package, but did not  
perform post-hoc tests. Other contributions showed examples with only  
one within-subjects factors. Finally, Achim Zeileis (thanks again!)  
helped me with the following approach.

1.  Use the nlme-Package and calculate a model with both within- 
subjects effects "Condition1" and "Condition2"

library(nlme)
d.lme <- lme(Result~ Condition1*Condition2,data=d,random= ~1 | Subject)

2. Finally, the multcomp-Package can be used to perform post-hoc  
tests for  "Condition1" and "Condition2"

library(multcomp)
print(summary(glht(d.lme, linfct=mcp(Condition1="Tukey"))))
print(summary(glht(d.lme, linfct=mcp(Condition2="Tukey"))))

My problems and questions are

1) When applying this solution to my collection of data I found  
several cases where the standard repeated-measures ANOVA showed a  
highly significant effect for both factors, e. g.

ano <- aov(d$wPatternPulseSNR~ d$Bedingung*d$Felder + Error(d$VPerson/ 
(d$Bedingung*d$Felder), data=d))

Error: d$VPerson
           Df Sum Sq Mean Sq F value Pr(>F)
Residuals 11 458.22   41.66

Error: d$VPerson:d$Bedingung
             Df Sum Sq Mean Sq F value    Pr(>F)
d$Bedingung  4 364.58   91.14  7.4429 0.0001140 ***
Residuals   44 538.81   12.25

Error: d$VPerson:d$Felder
           Df Sum Sq Mean Sq F value    Pr(>F)
d$Felder   5 464.17   92.83  8.3957 5.953e-06 ***
Residuals 55 608.16   11.06

but the multcomp-results indicated no significant post-hoc  
differences between any pair of values for both factors (here the  
values for the factor "Felder" as example)

Fit: lme.formula(fixed = wPatternPulseSNR ~ Bedingung * Felder, data  
= d,
     random = ~1 | VPerson)

Linear Hypotheses:
                    Estimate Std. Error z value Pr(>|z|)
Feld2 - Feld1 == 0  1.89748    0.84245   2.252    0.214
Feld3 - Feld1 == 0  1.41383    0.84245   1.678    0.546
Feld4 - Feld1 == 0  1.48945    0.84245   1.768    0.487
Feld5 - Feld1 == 0 -0.11133    0.84245  -0.132    1.000
Feld6 - Feld1 == 0  0.02472    0.84245   0.029    1.000
Feld3 - Feld2 == 0 -0.48366    0.84245  -0.574    0.993
Feld4 - Feld2 == 0 -0.40803    0.84245  -0.484    0.997
Feld5 - Feld2 == 0 -2.00882    0.84245  -2.385    0.162
Feld6 - Feld2 == 0 -1.87277    0.84245  -2.223    0.227
Feld4 - Feld3 == 0  0.07562    0.84245   0.090    1.000
Feld5 - Feld3 == 0 -1.52516    0.84245  -1.810    0.459
Feld6 - Feld3 == 0 -1.38911    0.84245  -1.649    0.566
Feld5 - Feld4 == 0 -1.60078    0.84245  -1.900    0.402
Feld6 - Feld4 == 0 -1.46473    0.84245  -1.739    0.506
Feld6 - Feld5 == 0  0.13605    0.84245   0.161    1.000
(Adjusted p values reported -- single-step method)


2) So my main question is whether I really applied the correct data  
analysis to the data? Perhaps the discrepancy between the aov-results  
and the lme-results indicate the need to perform the post-hoc tests  
correctly. On the other hand, the difference between a p-value of  
5.953e-06 (aov) and p>0.1 (for all pairwise comparisons) simply  
indicates, that I did something wrong...


Best wishes
Thomas


	[[alternative HTML version deleted]]

