[R-sig-eco] AIC / BIC vs P-Values in lmer
Chris Mcowen
chrismcowen at gmail.com
Thu Aug 5 15:27:34 CEST 2010
Hi Andrew,
Thanks very much for this, however, does this mean that the problems with interpreting the AIC value as explained by Phillip Dixon still apply?
> 1) the AIC calculated from the reml lnL only informs you about the fit of the random effects model.
> 2) the reml AIC can only be compared between models with the same fixed effects. Otherwise, the lnL is being calculated from different data (because different residuals with different X's).
If so is there a work around using binomial data in lmer?
Thanks
Chris
On 5 Aug 2010, at 14:22, Crowe, Andrew wrote:
Chris/Ben
The lack of effect of the REML parameter is simply explained by the fact you are fitting a binomial model. This causes the lmer call to default to a glmer call in which the REML parameter is ignored. I also note that you are specifying order/family in the random term, which I assume are the taxanomic definitions of family and order. As family is completey nested in order so that order:family is as unique as family, no additional variance is explained by order over family so I believe that you should just be able to specify (1|family) for your random intercept.
Regards
Andrew
Dr Andrew Crowe
Lancaster Environment Centre
Lancaster University
Lancaster LA1 4YQ
UK
________________________________
From: r-sig-ecology-bounces at r-project.org on behalf of Chris Mcowen
Sent: Thu 05/08/2010 2:04 PM
To: Ben Bolker
Cc: r-sig-ecology at r-project.org
Subject: Re: [R-sig-eco] AIC / BIC vs P-Values in lmer
I have just tried it with REML=FALSE and once again there is no difference in the AIC/BIC values between the two models? I have given two examples this time but have tried it with 10 models with no difference.
Thanks,
Chris
1
MODEL WITH REML=FALSE
> model01 <- lmer(threatornot~1+(1|order/family) + seasonality + pollendispersal + breedingsystem*fruit + habit + lifeform + woodyness, family=binomial,REML=FALSE )
Generalized linear mixed model fit by the Laplace approximation
Formula: threatornot ~ 1 + (1 | order/family) + seasonality + pollendispersal + breedingsystem * fruit + habit + lifeform + woodyness
AIC BIC logLik deviance
1399 1479 -683.6 1367
Random effects:
Groups Name Variance Std.Dev.
family:order (Intercept) 0.27526 0.52466
order (Intercept) 0.00000 0.00000
Number of obs: 1116, groups: family:order, 43; order, 9
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.384574 0.734960 0.523 0.60079
seasonality2 -1.127996 0.353013 -3.195 0.00140 **
pollendispersal2 0.693255 0.314600 2.204 0.02755 *
breedingsystem2 0.761067 0.493404 1.542 0.12296
breedingsystem3 1.226269 0.557236 2.201 0.02776 *
fruit2 1.047648 0.616723 1.699 0.08937 .
habit2 -1.146334 0.551682 -2.078 0.03772 *
habit3 -0.731207 0.872805 -0.838 0.40216
habit4 -0.190900 0.551427 -0.346 0.72920
lifeform2 -0.295342 0.182667 -1.617 0.10592
lifeform3 -0.376204 0.501825 -0.750 0.45345
woodyness2 0.006274 0.390241 0.016 0.98717
breedingsystem2:fruit2 -1.273811 0.651011 -1.957 0.05039 .
breedingsystem3:fruit2 -1.633424 0.744563 -2.194 0.02825 *
MODEL WITHOUT REML=FALSE
model126 <- lmer(threatornot~1+(1|order/family) + seasonality + pollendispersal + breedingsystem*fruit + habit + lifeform + woodyness, family=binomial)
Generalized linear mixed model fit by the Laplace approximation
Formula: threatornot ~ 1 + (1 | order/family) + seasonality + pollendispersal + breedingsystem * fruit + habit + lifeform + woodyness
AIC BIC logLik deviance
1399 1479 -683.6 1367
Random effects:
Groups Name Variance Std.Dev.
family:order (Intercept) 0.27526 0.52466
order (Intercept) 0.00000 0.00000
Number of obs: 1116, groups: family:order, 43; order, 9
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.384574 0.734960 0.523 0.60079
seasonality2 -1.127996 0.353013 -3.195 0.00140 **
pollendispersal2 0.693255 0.314600 2.204 0.02755 *
breedingsystem2 0.761067 0.493404 1.542 0.12296
breedingsystem3 1.226269 0.557236 2.201 0.02776 *
fruit2 1.047648 0.616723 1.699 0.08937 .
habit2 -1.146334 0.551682 -2.078 0.03772 *
habit3 -0.731207 0.872805 -0.838 0.40216
habit4 -0.190900 0.551427 -0.346 0.72920
lifeform2 -0.295342 0.182667 -1.617 0.10592
lifeform3 -0.376204 0.501825 -0.750 0.45345
woodyness2 0.006274 0.390241 0.016 0.98717
breedingsystem2:fruit2 -1.273811 0.651011 -1.957 0.05039 .
breedingsystem3:fruit2 -1.633424 0.744563 -2.194 0.02825 *
2
MODEL WITH REML=FALSE
> model02 <- lmer(threatornot~1+(1|order/family) + seasonality + woodyness, family=binomial,REML=FALSE )
Generalized linear mixed model fit by the Laplace approximation
Formula: threatornot ~ 1 + (1 | order/family) + seasonality + woodyness
AIC BIC logLik deviance
1395 1420 -692.6 1385
Random effects:
Groups Name Variance Std.Dev.
family:order (Intercept) 0.49348 0.70248
order (Intercept) 0.00000 0.00000
Number of obs: 1116, groups: family:order, 43; order, 9
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.6034 0.4227 1.427 0.15346
seasonality2 -1.1421 0.3453 -3.308 0.00094 ***
woodyness2 0.5113 0.2559 1.998 0.04572 *
MODEL WITHOUT REML=FALSE
model03 <- lmer(threatornot~1+(1|order/family) + seasonality + woodyness, family=binomial)
Generalized linear mixed model fit by the Laplace approximation
Formula: threatornot ~ 1 + (1 | order/family) + seasonality + woodyness
AIC BIC logLik deviance
1395 1420 -692.6 1385
Random effects:
Groups Name Variance Std.Dev.
family:order (Intercept) 0.49348 0.70248
order (Intercept) 0.00000 0.00000
Number of obs: 1116, groups: family:order, 43; order, 9
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.6034 0.4227 1.427 0.15346
seasonality2 -1.1421 0.3453 -3.308 0.00094 ***
woodyness2 0.5113 0.2559 1.998 0.04572 *
On 5 Aug 2010, at 13:51, Ben Bolker wrote:
Chris Mcowen <chrismcowen at ...> writes:
>
> Hi Philip,
>
> Thanks very much for this, i was completely unaware. I have read various
papers using lmer to calculate the
> AIC statistic and none have mentioned this?
>
> I have just run through a random section of my models with this correction,
however the AIC / BIC values are
> the same with the REML=F in and out?
>
> Chris
Try REML=FALSE instead ... ? (You may have 'F' set to a value
in your workspace.) Otherwise I would find it very odd that the
results are identical.
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
[[alternative HTML version deleted]]
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
More information about the R-sig-ecology
mailing list