[R-sig-ME] Very different results from lmer and MCMCglmm

Tue Jan 31 18:51:03 CET 2012

Hello, I have a dataset with outcomes {1, 2, 3, 4}. The outcome variable
is actually ordered categories, but as point of reference for
comparison, I analyzed it as numeric in lmer, and got these results:

Linear mixed model fit by REML 
Formula: rating ~ comp.f + grade.f + subject.f + obsord.f + (1 | obsid)
+      (1 | tid) + (1 | grade.f) + (1 | subject.f) + (1 | obsord.f) 
   Data: ratings.prin 
  AIC  BIC logLik deviance REMLdev
 6886 7058  -3416     6740    6832
Random effects:
 Groups    Name        Variance   Std.Dev.
 tid       (Intercept) 0.19082494 0.436835
 obsid     (Intercept) 0.10405718 0.322579
 subject.f (Intercept) 0.00075553 0.027487
 grade.f   (Intercept) 0.00075435 0.027465
 obsord.f  (Intercept) 0.00060346 0.024565
 Residual              0.24073207 0.490645
Number of obs: 4253, groups: tid, 245; obsid, 94; subject.f, 5; grade.f,
5; obsord.f, 4

Fixed effects:
             Estimate Std. Error t value
(Intercept)  3.261329   0.140592  23.197
comp.f2     -0.095729   0.033461  -2.861
comp.f3     -0.061422   0.033316  -1.844
comp.f4     -0.144613   0.033364  -4.334
comp.f5     -0.059794   0.033599  -1.780
comp.f6     -0.074454   0.033249  -2.239
comp.f7     -0.325454   0.033274  -9.781
comp.f8     -0.186724   0.033187  -5.626
comp.f9     -0.320803   0.033741  -9.508
comp.f10    -0.226328   0.034056  -6.646
grade.f2    -0.203406   0.140249  -1.450
grade.f3    -0.227049   0.134389  -1.689
grade.f4    -0.377642   0.137710  -2.742
grade.f5    -0.225643   0.140196  -1.609
subject.f2  -0.009939   0.053291  -0.187
subject.f3   0.289519   0.061324   4.721
subject.f4  -0.223719   0.107737  -2.077
subject.f5  -0.025963   0.073520  -0.353
obsord.f2    0.004840   0.038436   0.126
obsord.f3    0.112110   0.052707   2.127
obsord.f4    0.156406   0.078614   1.990

These results seem somewhat reasonable to me. But when I analyze the
very same dataset using the same model in MCMCglmm I get very different
results:

glme5 <- MCMCglmm(rating.o ~ comp.f + grade.f + subject.f + obsord.f ,
                  prior=list(R=list(V=1, fix=1), G=list(G1=list(V=1,
nu=0), G2=list(V=1, nu=0), G3=list(V=1, nu=0), G4=list(V=1, nu=0),
G5=list(V=1, nu=0) )),
               random = ~tid + obsid + grade.f + subject.f + obsord.f ,
               family = "ordinal",
               nitt=100000,
               data = ratings.prin)

 Iterations = 3001:99991
 Thinning interval  = 10
 Sample size  = 9700 

 DIC: 5701.873 

 G-structure:  ~tid

    post.mean l-95% CI u-95% CI eff.samp
tid     2.423    1.821    3.063     2759

               ~obsid

      post.mean l-95% CI u-95% CI eff.samp
obsid     1.521   0.7707    2.331     5227

               ~grade.f

        post.mean  l-95% CI  u-95% CI eff.samp
grade.f  95365148 2.234e-17 104888830     2296

               ~subject.f

          post.mean  l-95% CI  u-95% CI eff.samp
subject.f   7.5e+07 1.502e-17 101313849     3950

               ~obsord.f

         post.mean  l-95% CI u-95% CI eff.samp
obsord.f 122278523 2.079e-17 64065615     3851

 R-structure:  ~units

      post.mean l-95% CI u-95% CI eff.samp
units         1        1        1        0

 Location effects: rating.o ~ comp.f + grade.f + subject.f + obsord.f 

             post.mean   l-95% CI   u-95% CI eff.samp    pMCMC    
(Intercept)  1.430e+02 -2.218e+04  1.781e+04    10178 0.607629    
comp.f2     -3.448e-01 -5.854e-01 -1.161e-01     6220 0.004124 ** 
comp.f3     -2.219e-01 -4.527e-01  1.402e-02     6328 0.064124 .  
comp.f4     -5.166e-01 -7.459e-01 -2.831e-01     6454 0.000206 ***
comp.f5     -2.087e-01 -4.431e-01  2.333e-02     6338 0.084536 .  
comp.f6     -2.692e-01 -5.091e-01 -4.112e-02     6290 0.024948 *  
comp.f7     -1.163e+00 -1.403e+00 -9.395e-01     4027  < 1e-04 ***
comp.f8     -6.682e-01 -9.011e-01 -4.368e-01     5448  < 1e-04 ***
comp.f9     -1.157e+00 -1.392e+00 -9.171e-01     4253  < 1e-04 ***
comp.f10    -8.167e-01 -1.056e+00 -5.742e-01     6152  < 1e-04 ***
grade.f2    -2.417e+00 -7.966e+03  8.888e+03    13314 0.396082    
grade.f3     1.304e+02 -7.486e+03  9.484e+03    10314 0.342062    
grade.f4    -1.684e+02 -9.879e+03  6.926e+03    12352 0.283711    
grade.f5     1.218e+02 -8.380e+03  7.895e+03     8740 0.351546    
subject.f2  -9.163e+01 -7.562e+03  7.806e+03    12224 0.930309    
subject.f3   1.699e+01 -7.411e+03  8.238e+03    12320 0.344536    
subject.f4   3.477e+01 -9.427e+03  7.519e+03    13106 0.372165    
subject.f5  -1.203e+02 -7.618e+03  8.837e+03     9071 0.848247    
obsord.f2   -5.860e+01 -7.058e+03  5.605e+03     9290 0.819794    
obsord.f3   -9.302e+01 -5.852e+03  5.641e+03     7386 0.332990    
obsord.f4   -1.243e+02 -6.891e+03  6.093e+03    10073 0.343299    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

 Cutpoints: 
                         post.mean l-95% CI u-95% CI eff.samp
cutpoint.traitrating.o.1     3.309    3.101    3.517    172.5
cutpoint.traitrating.o.2     6.790    6.552    7.056    150.1

Obviously, something has gone kablooey here. The confidence intervals
for the grade, subject and obsord random effects range over 25 orders of
magnitude, and the fixed effects are also extremely large (but with
correspondingly large standard errors). The intercept is 143, while the
outcomes only range between 1 and 4. Can anyone tell me what I have
screwed up here?

TIA.

-- 
Stuart Luppescu -=- slu .at. ccsr.uchicago.edu        
University of Chicago -=- CCSR 
才文と智奈美の父 -=-    Kernel 3.2.1-gentoo-r2                
To paraphrase provocatively, 'machine learning is
 statistics minus any checking of models and
 assumptions'.    -- Brian D. Ripley (about the
 difference between machine learning and      
 statistics)       useR! 2004, Vienna (May 2004)