[R-sig-ME] How sinful is it to...

[Ben Bolker]

David Sidhu <dsidhu at ...> writes:

> 
> Hi Everyone!
> 
> I am conducting a mixed model logistic regression analysis 
> of some simple experimental data in which
> participants made dichotomous choices. My independent 
> variable has two levels. What I am really
> interested in is the ability of the independent variable 
> to predict the dependent variable, with the
> random effects in there simply to allow the best test of this.
> 
> I am starting from the point of view that I should “keep it maximal”. 
> That being said my two questions are:
> 
>   *   Is there something horrible about keeping in 
> random effects that are highly (or perfectly!) correlated
> with one another?
>      *   If so…would you retain the term that gives you 
> the best model, based on AIC or some other value?
>   *   Should you always remove random effects that have 
> 0 variance associated with them?
> 
> It seems like the intuitive answer to both of these is yes. 
> But in this paper I am working on, there are several
> experiments, and for simplicity’s sake it would be much easier 
> to always just have the maximally
> complex random effects structure that managed to converge, 
> instead of giving details in each case about
> which terms were removed and why. But is there something
>  horrible wrong with that??

   I don't think so, but be aware that this is very much an open
question.  At one level, the perfectly correlated (or zero-variance)
solution is the correct answer to the question you posed with your
model ("what set of parameters maximizes the likelihood of this
statistical model"?)  The things to be concerned about would be:

* this suggests the model is overfitted (but it's not obvious that
removing these terms post hoc, introducing a model selection step into
the statistical procedure, will automatically solve the problem

http://arxiv.org/abs/1506.04967

(Example 1) "Fortunately and interestingly, none of the analyses impacted the
statistical inference about fixed effects in these experiments.
Again, the different model
specifications reported in this section were of no consequence for the
significance or interpretation
of fixed effects, but they led to inappropriate conclusions about the
correlations between variance
components.