[R-sig-ME] Why does anova.merMod use refit?

[Ben Bolker]

Henrik Singmann <henrik.singmann at ...> writes:

 
> I am sorry for the misleading title of my mail. I didn't want to
> discuss the utility of having LRT tests done with REML versus not
> REML estimates, I think this doesn't deserve much discussion (i.e.,
> REML + LRT = bad).
 
> The point I wanted to make is just that it is relatively uncommon
> that an anova method is time consuming (which it may be in case of
> refit) and furthermore it is somewhat unsatisfactory that anova does
> not allow to pass the really nice control options to the
> optimizer. Obviously one can simply circumvent this issue by fitting
> the model directly with lmer and REML = FALSE and pass the correct
> optimizer arguments and then use this model as argument to anova.
 
> However, the optimization envoked by refit may also fail to converge
> in the number of default maxfun steps simply if there are many
> observations. This does not need to be due to a misspecified
> model. In fact in my real data, the warning message said exactly
> this (and came through anova): "maxfun < 10 * length(par)^2 is not
> recommended". And there is no way to change maxfun if refit is
> envoked.
 
> Hence my pledge for a warning instead of calling refit automatically.
> 
> Best,
> Henrik
 
> PS: I realize that this problem only appeared because I stupidly
> tried to compare to REML LMMs with anova which I shouldn't have
> done...

  I think there are some reasonable design considerations here,
not obvious what the best answer(s) is/are.

1. one way or the other we should prevent users from running
anova() on REML-fitted models *with different fixed effects* (arguably
there is a sensible use case for using anova() on models that
differ only in their random effects
  1a. should we try to do this automatically or should we throw
an error and hint to them how best to achieve their goal?

2 refit should allow appropriate optimization control parameters to be set
  2a a model refit should always inherit the optimization control parameters
from its parent model

3 we should try not to make too many backward-incompatible changes
(i.e. even if the current behaviour is weird we should make changes
carefully/gradually so as not to disturb users who have adapted to it)

  Point #2 is especially relevant; the design goal is to have #2a
be true, but it might not be.  Examples of where it is false, especially
minimalistic ones that can be run and tested as part of unit tests,
are especially welcome.

  Ben Bolker