I'm puzzled as to what the best mixed-effect model is for my data (crossed
case, from experiment with between-subject factors: L1, pro ciency (+
possibly Age and Sex) and within-subject factors: Word.Order, Priming,
Relation).

The best model in terms of LogLik and AIC (but not BIC) values is
dat.lmer11, but it does not return significant p-values for the 3-way
interactions.

> dat.lmer11 = lmer(-1000 * Target.RTinv ~ (1|Subject) + (1|Target) +
Proficiency + (Word.Order + L1 + Priming)^3, dat, REML=TRUE)
> dat.lmer12c = lmer(-1000 * Target.RTinv ~ (1|Subject) + (1|Target) +
Proficiency + Priming + Word.Order*L1, dat, REML=TRUE)
> anova(dat.lmer11, dat.lmer12c)
    Data: dat
    Models:
    dat.lmer12c: -1000 * Target.RTinv ~ (1 | Subject) + (1 | Target) +
Proficiency +
    dat.lmer12c:     Priming + Word.Order * L1
    dat.lmer11: -1000 * Target.RTinv ~ Proficiency + (Word.Order + L1 +
Priming)^3 +
    dat.lmer11:     (1 | Subject) + (1 | Target)
                Df    AIC    BIC  logLik  Chisq Chi Df Pr(>Chisq)
    dat.lmer12c 11 1790.8 1868.1 -884.39
    dat.lmer11  16 1788.5 1901.0 -878.24 12.295      5    0.03097 *
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


> pvals.fnc(dat.lmer11)
    $fixed
                                             Estimate MCMCmean HPD95lower
HPD95upper  pMCMC Pr(>|t|)
    (Intercept)                               -0.1546  -0.1542    -0.7731
  0.4251 0.5970   0.6410
    Proficiency                               -0.9625  -0.9631    -1.5574
 -0.3691 0.0026   0.0034
    Word.OrderModHead                         -0.1590  -0.1590    -0.1898
 -0.1290 0.0001   0.0000
    L1German                                   0.1710   0.1712    -0.0472
  0.3795 0.1114   0.1473
    L1Spanish                                  0.2622   0.2602     0.0787
  0.4440 0.0082   0.0082
    PrimingPrMod                              -0.0073  -0.0071    -0.0310
  0.0161 0.5538   0.5467
    Word.OrderModHead:L1German                 0.0665   0.0665     0.0276
  0.1063 0.0010   0.0007
    Word.OrderModHead:L1Spanish                0.0962   0.0965     0.0573
  0.1348 0.0001   0.0000
    Word.OrderModHead:PrimingPrMod            -0.0446  -0.0449    -0.0771
 -0.0108 0.0070   0.0084
    L1German:PrimingPrMod                     -0.0414  -0.0416    -0.0802
 -0.0016 0.0400   0.0382
    L1Spanish:PrimingPrMod                    -0.0253  -0.0252    -0.0642
  0.0157 0.2220   0.2177
    Word.OrderModHead:L1German:PrimingPrMod    0.0458   0.0461    -0.0081
  0.1013 0.0946   0.0968
    Word.OrderModHead:L1Spanish:PrimingPrMod   0.0094   0.0090    -0.0441
  0.0661 0.7512   0.7402


An alternative model with sub-optimal fit returns significant p-values for
all the fixed-effect factors.

> pvals.fnc(dat.lmer12c)
    $fixed
                                Estimate MCMCmean HPD95lower HPD95upper
 pMCMC Pr(>|t|)
    (Intercept)                  -0.1391  -0.1404    -0.7791     0.4525
0.6404   0.6751
    Proficiency                  -0.9616  -0.9607    -1.5681    -0.3417
0.0040   0.0035
    PrimingPrMod                 -0.0409  -0.0409    -0.0523    -0.0300
0.0001   0.0000
    Word.OrderModHead            -0.1810  -0.1812    -0.2067    -0.1552
0.0001   0.0000
    L1German                      0.1508   0.1513    -0.0729     0.3621
0.1694   0.2000
    L1Spanish                     0.2503   0.2504     0.0643     0.4314
0.0112   0.0112
    Word.OrderModHead:L1German    0.0891   0.0894     0.0638     0.1184
0.0001   0.0000
    Word.OrderModHead:L1Spanish   0.1003   0.1004     0.0725     0.1275
0.0001   0.0000


Could somebody advise as to what I should do, please?

Many thanks in advance.

Cecile

	[[alternative HTML version deleted]]

