[R-sig-ME] Advice on comparing non-nested random slope models
Craig DeMars
cdemars at ualberta.ca
Sun Mar 26 22:15:16 CEST 2017
Hello,
This is a bit of a follow-up to a question last week on selecting among
GLMM models. Is there a recommended strategy for comparing non-nested,
random slope models? I have seen a similar question posted here
http://stats.stackexchange.com/questions/116935/comparing-non-nested-
models-with-aic but it doesn't seem to answer the problem - and maybe there
is no "answer". Zuur et al. (2010) discuss model selection but only in a
nested framework. Bolker et al. (2009) suggest AIC can be used in GLMMs but
caution against boundary issues and don't specifically mention any issues
with comparing different random effects structures (as Zuur does).
The context of my question comes from an analysis where we have 5 *a priori*
hypotheses describing different climate effects on juvenile recruitment in
an ungulate species. The data set has 21 populations (or herds) with
repeated annual measurements of recruitment and the climate variables
measured at the herd scale. To generate SE's that reflect herd as the
sampling unit, explanatory variables are specified as random slopes within
herd (as recommended by Schielzeth & Forstmeier 2009; Year is also
specified as a random intercept). Because there are only 21 herds, models
are fairly simple with only 2-3 explanatory variables (3 may by pushing
it...????). I can't post the data but it isn't really relevant to the
question (I think).
Initially, we looked at AIC to compare models. At the bottom of this
email, I have pasted the output from two models, each representing separate
hypotheses, to illustrate "the problem". The first model yields an AIC
value of 2210.7. The second model yields an AIC of 2479.5. Using AIC, Model
1 would be the "best" model. However, examining the parameter estimates
within each model makes me think twice about declaring Model 1 (or the
hypothesis it represents) as the most parsimonious explanation for the
data. In Model 1, two of the thee fixed effects estimates have small effect
sizes and all estimates are "non-significant" (if one considers
p-values....). In Model 2, two of the three fixed effect estimates have
larger effect sizes are would be considered "significant. Is this an
example of the difficulty in using AIC to compare non-nested mixed
models.....or am I missing something in my interpretation? I haven't come
across this type of result when model selecting among GLMs.
Any suggestions on how best to compare competing hypotheses represented by
non-nested GLMMs? Should one just compare relative effect sizes of
parameter estimates among models?
Any help would be appreciated.
Thanks,
Craig
*Model 1:*
Generalized linear mixed model fit by maximum likelihood (Laplace
Approximation) ['glmerMod']
Family: binomial ( logit )
Formula: (Calves/Cows) ~ spr.indvi.ab + green.rate.ab + trend + (1 | Year)
+ (spr.indvi.ab + green.rate.ab + trend | Herd)
Data: bou.dat
Weights: Cows
*AIC * BIC logLik deviance df.resid
*2210.7* 2265.0 -1090.3 2180.7 262
Scaled residuals:
Min 1Q Median 3Q Max
-3.8700 -1.0800 -0.1057 1.0405 6.8353
Random effects:
Groups Name Variance Std.Dev. Corr
Year (Intercept) 0.10517 0.3243
Herd (Intercept) 0.29832 0.5462
spr.indvi.ab 0.04331 0.2081 0.38
green.rate.ab 0.03741 0.1934 0.68 0.62
trend 0.62661 0.7916 -0.59 0.20 -0.46
Number of obs: 277, groups: Year, 22; Herd, 21
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.62160 0.15798 -10.265 <2e-16 ***
spr.indvi.ab 0.04019 0.09793 0.410 0.682
green.rate.ab 0.04704 0.05555 0.847 0.397
trend -0.29676 0.23092 -1.285 0.199
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) spr.n. grn.r.
spr.indvi.b -0.113
green.rat.b 0.347 0.438
trend -0.606 0.349 -0.200
*Model 2:*
Generalized linear mixed model fit by maximum likelihood (Laplace
Approximation) ['glmerMod']
Family: binomial ( logit )
Formula: (Calves/Cows) ~ win.bb + tot.sn.ybb + trend + (1 | Year) + (win.bb
+ tot.sn.ybb | Herd)
Data: bou.dat
Weights: Cows
* AIC* BIC logLik deviance df.resid
*2479.5 * 2519.4 -1228.8 2457.5 266
Scaled residuals:
Min 1Q Median 3Q Max
-4.5720 -1.1801 -0.1364 1.3704 8.3271
Random effects:
Groups Name Variance Std.Dev. Corr
Year (Intercept) 0.10694 0.3270
Herd (Intercept) 0.13496 0.3674
win.bb 0.05351 0.2313 -0.13
tot.sn.ybb 0.06200 0.2490 0.23 0.34
Number of obs: 277, groups: Year, 22; Herd, 21
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.851656 0.127702 -14.500 < 2e-16 ***
win.bb -0.364019 0.101386 -3.590 0.00033 ***
tot.sn.ybb 0.275271 0.118111 2.331 0.01977 *
trend -0.007568 0.115706 -0.065 0.94785
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) win.bb tt.sn.
win.bb 0.048
tot.sn.ybb 0.269 0.083
trend -0.242 -0.269 -0.131
--
Craig DeMars, Ph.D.
Postdoctoral Fellow
Department of Biological Sciences
University of Alberta
Phone: 780-221-3971 <(780)%20221-3971>
[[alternative HTML version deleted]]
More information about the R-sig-mixed-models
mailing list