[R-sig-ME] GLM,GLMM,CAIC,mcmcGLMM - AIC model selction

[Chris Mcowen]

Hi Andrew,

I appreciate that you can not compare the AIC values of GLM and GLMM, however  i was looking at AIC differences between a model set for each method  i.e

GLMM

Model		AIC		AIC DIFF
Model 1		100		0
Model 2		110		10
Model 3		120		20

GLMM

Model		AIC		AIC DIFF
Model 1		300		0
Model 2		307		7
Model 3		317		17


This is similar ( although simplified) to the results i have ( as shown by the links in the original post), it is this that i am confused about. Furthermore the same pattern showed when using CAIC and mcmcGlmm. I am wondering if this implies that for the purpose of model selection using GLMM and GLM produces similar results? 

Chris

 		
On 13 Jan 2011, at 13:28, Andrew Kosydar wrote:

Hi Chris,

My understanding is that comparing GLM to GLMM right now is not
possible via multimodel model inference (MMI) because the number of
degrees of freedom for random effects are estimated to be equal to
fixed effects and this is generally considered to be inaccurate.  As a
result, the AICc values produced are not comparable.

Regarding the remainder of your study and phylogenetic corrections, I
am very interested to know your results as I have a similar dilemma
with one of my studies that I am reworking for publication.

Best of luck,

Andrew


-- 
Andrew Kosydar, PhD
drewdogy at uw.edu








On Thu, Jan 13, 2011 at 3:56 AM, Chris Mcowen <chrismcowen at gmail.com> wrote:
> Dear list,
> 
> I am modelling the effect of various life history traits of species against their extinction rating.
> 
> My data has a phylogentic signal (see table below) so to be statistically correct i worked with phylogentic independent contrasts
> 
> From Purvis et al., 2000 "phylogenetic analyses were necessary because of the pseudoreplication and, hence, elevated type I error rates that result from treating species as independent points when relevant variables show a phylogenetic pattern"
> 
> Variable                                        λ ( Pagel)
> IUCN extinction risk            0.47
> Breeding system                 0.99
> Endosperm                               0.96
> Floral symmetry                 0.93
> Fruit                                   0.97
> Pollen dispersal                        0.99
> Seasonality                             0.52
> Storage organ                   0.85
> Woodyness                               0.61
> 
> There are various ways of doing this one method is to use the package CAIC (generated using compara- tive analysis by independent contrasts)  which utilizes the phylogeny to generate the independent contrasts. However as i was using it i found this: from Sodhi et al., 2008
> 
>> It was necessary to decompose the variance across species by coding the random-effects error structure of the GLMM as a hierarchical taxonomic (class/order/ family) effect (Blackburn & Duncan, 2001). We had insufficient replication within genera to include the genus in the nested random effect. Our method is more appropriate than the independent-contrasts approach (Purvis et al., 2000) in situations where a complete phylogeny of the study taxon is unavailable, when categorical variables are included in the analysis, and when model selection, rather than hypothesis testing, is the statistical paradigm being used.
> 
> So i gave this a go - using GLMM and setting the order / family as random effects.
> 
> I then came across mcmcGLMM which allows the use of a phylogentic tree to deal with the phylogenetic structure of the data set.
> 
> So i gave this a go as well!
> 
> Finally to be consistent i ran the model with no phylogenetic control - using GLM.
> 
> My criteria for model selection was that of Burnham and Anderson - using AIC and AICML etc to select the "most likely" set of models.
> 
> Interestingly i found that using all four methods the same model (based on AIC difference) was selected as the most likely? Furthermore, the pattern of AIC differences across the models reflected each other.
> 
> I was wondering why this maybe? Does anybody know of similar studies comparing model selection from GLM and GLMM?
> 
> I apologise if this is not the correct forum to post this question.
> 
> I have attached links to two graphs that explain my question.
> 
> Thanks Chris
> 
> LINKS
> 
> https://files.me.com/chrismcowen/hmptnq
> 
> files.me.com/chrismcowen/t3xvk5
> 
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>