[R-sig-ME] Comparing mixed models

Wed May 11 07:39:25 CEST 2016

Dear Jean-Philippe,

There are some papers that deal with the special case that the variance of  
an experimental design random term becomes negative due to a negative  
intraclass correlation. In old ANOVA models this could be detected as  
negative variance (this term will earn head shaking...), whereas in mixed  
models, where the design term is modeled at the random level, this is  
often not detectable because the design term variance may just be fixed at  
zero / converge to zero (if restrained to be positive). As a consequence,  
it happens that people tend to remove design terms from their models  
(because a zero variance random term clearly does not improve the model)  
and make inferences about, let's say treatments, based on observational  
rather than experimental units (that would only be represented by  
including the experimental design term) and this can lead to unrepeatable  
and overconfident inferences.

This problem cannot always be simply accounted for by leaving the random  
design term with a zero variance in the model. For example asreml-R does  
not account for zero-variance terms in F-tests (the denominator degrees of  
freedom inflate to observational level numbers), not sure what happens in  
lme4 / nlme models.

Here are some references about this very special topic that only covers  
the issue of zero-variance design terms that may in fact be negative, and  
how the experimental design can be accounted for at the residual level  
(with the associated consequences on prediction ability) in alternative to  
having zero-variance random terms:

Nelder, J. A. 1954. The interpretation of negative components of variance.  
Biometrika 41:544-548.

Wang, C. S., B. S. Yandell, and J. J. Rutledge. 1992. The dilemma of  
negative analysis of variance estimators of intraclass correlation.  
Theoretical and Applied Genetics 85:79-88.

Pryseley, A., C. Tchonlafi, G. Verbeke, and G. Molenberghs. 2011.  
Estimating negative variance components from Gaussian and non-Gaussian  
data: A mixed models approach. Computational Statistics & Data Analysis  
55:1071-1085.

I hope that is not too special case for your question, but I think it is a  
very important case for making inferences that account for an experimental  
design, i.e., when a non-significant random term should be left in the  
model.

Best,
Paul

On Wed, 11 May 2016 05:52:24 +0300, Jean-Philippe Laurenceau  
<jlaurenceau at psych.udel.edu> wrote:

> Dear Ben et al.--I agree with the general practice of trying to estimate  
> and retain as many random effects as possible (without estimation  
> issues) in a mixed model. However, I was wondering whether anyone had  
> some references recommending or arguing for this approach. I am aware of  
> a paper on this topic with some simulation work by Barr et al. (2013;  
> Journal of Memory and Language), but I would be interested in whether  
> there are others. Thanks, J-P
>
> Jean-Philippe Laurenceau, Ph.D.
> Department of Psychological & Brain Sciences
> University of Delaware
>
>
> -----Original Message-----
> From: R-sig-mixed-models  
> [mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of Ben Bolker
> Sent: Saturday, May 7, 2016 11:35 AM
> To: Carlos Barboza <carlosambarboza at gmail.com>
> Cc: r-sig-mixed-models at r-project.org
> Subject: Re: [R-sig-ME] Comparing mixed models
>
>   My only other comment would be that my standard approach would be to  
> retain all random effects in the model unless they are causing  
> difficulty in model fitting -- this depends on your goal  
> (confirmation/testing, prediction, exploration)
>
> On Sat, May 7, 2016 at 11:26 AM, Carlos Barboza  
> <carlosambarboza at gmail.com>
> wrote:
>
>> Dear Dr. Ben Bolker
>>
>> My name is Carlos Barboza and I am a Marine Biologist from the Rio de
>> Janeiro University, Brazil. First it's a pleasure to again have the
>> opportunity to send you a message.The reason for it is a simple doubt:
>> Can I compare AIC from:
>>
>> 1. glmmADMB: Density ~ 1 + 1|Site
>>
>> 2. glmmADMB: Density ~ Sector + 1|Site + Cage
>>
>> Note that they have different random and fixed structures. I know that
>> this is not the best choice to model selection but, I think that the
>> AIC values can be compared.
>>
>> thank you very much for your attention
>>
>>
>>   is Cage a random effect?  Are you intentionally leaving out the
>> intercept in the second case (it will be included anyway unless you
>> use -1)?  In any case, I don't see any obvious reason you can't
>> compare AIC values; see
>>
>> https://rawgit.com/bbolker/mixedmodels-misc/master/glmmFAQ.html#can-i-
>> use-aic-for-mixed-models-how-do-i-count-the-number-of-degrees-of-freed
>> om-for-a-random-effect
>>
>>   Follow-ups to r-sig-mixed-models at r-project.org, please ...
>>
>> sorry, yes, cage was included only to examplify a different random
>> structure in the second case...it should be coded (1|Site) + (1|Cage)
>> yes, I know that the intercept will be included in the second model
>>
>> it's an example of comparing AIC values from mixed models with
>> different fixed and random structures:
>>
>> 1. Density ~ 1 + 1|Site
>>
>> 2. Density ~ Sector + 1|Site + 1|Cage
>>
>> comparing AIC...I beleive that both values can be compared
>>
>> again, thank you very much for your very fast message
>>
>>
>>
>>
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list  
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

-- 
Paul V. Debes
DFG Research Fellow

Division of Genetics and Physiology
Department of Biology
University of Turku
PharmaCity, 7th floor
Itainen Pitkakatu 4
20014 Finland

Email: paul.debes at utu.fi