[R-sig-ME] AIC Comparison for MLM with Different Distributions

[Kate R]

Hi all,

Thank you in advance for your time and consideration! I am a
non-mathematically-inclined graduate student in communication just learning
multilevel modeling.

I am trying to compare the AIC for 5 different models:


   1. model.mn5 <- lmer(anxious ~ num.cm + num.pmc + (1|userid), data = df,
   REML = F)
   2. model.mn5.log <- lmer(log(anxious) ~ num.cm + num.pmc + (1|userid),
   data = df, REML = F)
   3. model.mn5.gamma.log <- glmer(anxious ~ num.cm + num.pmc + (1|userid),
   data = df, family = Gamma(link="log"))
   4. model.mn5.gamma.id <- glmer(anxious ~ num.cm + num.pmc + (1|userid),
   data = df, family = Gamma(link="identity"))
   5. model.ord5 <- clmm(anxious ~ num.cm + num.pmc + (1|userid), data =
   df, na.action = na.omit)

(num.cm is the group mean and num.pmc is the group-mean-centered score of
the predictor)

Despite many posts on various help forums, I understand that it's possible
to compare non-nested models with different distributions as long as all
terms, including constants, are retained (i.e. see Burnham & Anderson, Ch
6.7 <https://www.springer.com/gp/book/9780387953649>), but that different R
packages or model classes might handle constants differently or use
different algorithms (see point 7 <https://robjhyndman.com/hyndsight/aic/>),
thus making it difficult to directly compare AIC values. To avoid
this non-comparability pitfall, it was suggested in one post to calculate
your own log-likelihood (though I'm having trouble finding this post again).

Please could you help with the following:

   - What is the best practice for comparing the AICs for these 5 models?
   - What is the R-code for manually calculating the log-likelihood and/or
   the AIC to retain all terms, including constants?
   - Can you compare ordinal models (clmm) with the continuous models?
   - Do you recommend any other methods and/or packages for comparing
   models with different distributions and/or links?

Many thanks in advance for your time and consideration! I greatly
appreciate any suggestions.

Kind regards,
K

	[[alternative HTML version deleted]]