[R-sig-ME] Model comparison using BIC, AIC, -2Log

[Rune Haubo]

Hi Luis

I largely agree with Mike's answer and have the following additional
comments: The decision of whether a variable is taken as fixed or
random often rests on subject specific matter. An important question
is: Can the levels of the variable be considered as coming from a
normal distribution? But other aspects also play a role, such as the
number of realized levels of the variable (with only few levels, it
will often be appropriate to treat the variable as fixed anyhow). The
models rests on different distributional assumptions, so the decision
is often based on weighing the appropriateness of these assumptions.

To give more specific advise on the actual model comparison (ignoring
the question of the appropriateness of the comparison), it matters
whether you are thinking in terms of linear mixed models or
generalized linear mixed models. In the former case assuming you have
only one random effect and assuming lme is sufficient, you can do

fm.lme <- lme(....)
fm.lm <- lm(...)
anova(fm.lme, fm.lm)

If you are thinking in terms of generalized linear mixed models, and
you are using lmer, then maybe you can use something like

deviance(fm.lmer <- lmer(...))
deviance(fm.glm <- glm(...))

however, the reference distribution for the difference in deviance
depends on the actual body of the function calls.

Regards
Rune

2008/7/4 Luis Orlindo Tedeschi <luis.tedeschi at gmail.com>:
> Thanks Mike... and I thought it would have a single answer... I glanced
> over the link you provided; it will take me some time to digest it. My
> current problem is comparing a model with variable A as random effect vs
> a model with variable A as fixed effect. It gets vary confusing. Thanks
> again. Luis
>
> On Fri, 2008-07-04 at 09:28 +0100, Mike Dunbar wrote:
>> Dear Luis
>>
>> It is not necessarily straightforward but there is alot of information out there that can help you. Take a look at http://wiki.r-project.org/rwiki/doku.php?id=guides:lmer-tests and also look through the archives of this list, e.g. the thread entitled "[R-sig-ME] interpreting significance from lmer results for dummies (like me)"
>>
>> regards
>>
>> Mike
>>
>>
>> >>> Luis Orlindo Tedeschi <luis.tedeschi at gmail.com> 03/07/2008 22:23 >>>
>> Folks; I have a quick question about model comparison. Is it ok to use
>> BIC/AIC/-2log to compare models with different fixed and random effects
>> and even different var-(co)var structure? How can I accomplish this
>> using R? Will Anova do the correct comparison of different models?
>> Thanks in advance. Luis
>>
>> --
>> Luis Orlindo Tedeschi <luis.tedeschi at gmail.com>
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>>
> --
>
> +----------------------------------------------------+
>              Luis O. Tedeschi, PhD, PAS
>                 Assistant Professor
>                 Texas A&M University
>
> 230 Kleberg Center               p. (+1) 979-845-5065
> 2471 TAMU                        f. (+1) 979-845-5292
> College Station, TX 77843-2471
>
> http://nutritionmodels.tamu.edu
> http://nutr.tamu.edu
> http://people.tamu.edu/~luis.tedeschi
> +----------------------------------------------------+
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>



-- 
Rune Haubo Bojesen Christensen

Master Student, M.Sc. Eng.
Phone: (+45) 30 26 45 54
Mail: rhbc at imm.dtu.dk, rune.haubo at gmail.com

DTU Informatics, Section for Statistics
Technical University of Denmark, Build.321, DK-2800 Kgs. Lyngby, Denmark