[R] Using MCMC sampling to estimate p values with a mixed model

[Ben Bolker]

Woodcock, Helena <H.E.Woodcock <at> liverpool.ac.uk> writes:

> Apologies if this is a silly question but I am a student 
> and this is my first time using R so I am still trying to
> educate myself on commands, models e.t.c
> 
> I have a mixed model with four dichotomous fixed factors and 
> subject as a random factor (as each person
> completed four vignettes, with factors crossed across vignettes).
> 
> I have run an lmer model and used the Monte Carlo method to 
> see if there are any significant main effects or
> interactions. However, when I looked at the p values 
> some are showing as significant although the F value
> is less than 1. Is it possible to have a significant 
> effect with an F value below 1?.
> 
> I have a sample size of 150 and have read that the 
> pMCMC values can be anti-conservative so wonder if it is
> because my sample size may be too small?.
> 

  It's hard to know exactly without more details/seeing the data;
it does sound suspicious.
  Unless there's something you haven't told us, it sounds like
this model is fairly close to a classical experimental design
(randomized block), so I would guess that the answers should (?) be
fairly close to the classical ones.  Have you tried fitting with
lme (from the nlme package) and seeing what it guesses for denominator
degrees of freedom, or working out appropriate denominator degrees
of freedom yourself?
  
  I would recommend that you send follow-ups to 
r-sig-mixed-models at r-project.org ...