[R] Question: Do I need to set refit=FALSE when testing for random effects with anova()?

[Christian Brauner]

Hello,

I am currently testing whether I should include certain random effects
in my lmer model or not. I use the anova function for that. My
procedure so far is to fit the model with a function call to lmer()
with REML=TRUE (the default option). Then I call anova() on the two
models where one of them does include the random effect to be tested
and the other one does not. However, it is well known that the anova()
function refits the models with ML (new versions also output a warning
that they do so). But in the new version of anova() you can prevent
anova() from doing so by setting the option refit=FALSE. In order to
test for random effects should I set refit=FALSE in my call to anova()
or not? If I do set refit=FALSE the p-values tend to be lower.
(Additional question: Are the p-values calculated by anova()
anti-conservative when I set refit=FALSE?)

Thanks for any help