[R-sig-ME] model selection methodology

[David Duffy]

Hi.

I suspect we would need more information. A real or simulated dataset of the correct form would be useful. In fact, if you could generate multiple such 
datasets, you could answer your own question about testing.

> I want to test whether a treatment (a) has an effect 
> on a dependent variable (y), so I built a full model :

> m1 <- glmmTMB(y ~ a + offset(log(b)) + (1|ID), data=x, 
>              ziformula =  ~ a,  family="poisson")

> I used the dredge function to generate a model selection table and the top
> model did not include the treatment (a) in the conditional model nor in the
> ZI model.

So. are there multiple records for each ID, or are you looking at both extra-poisson variation
and zero-inflation?

> My usual way to proceed is to run diagnostics for the selected model and
> conclude that the effect of treatment on my dependent variable is not
> statistically significant.

> My questions are:
> 1- is this the right way to proceed or should I check the diagnostics plots
> BEFORE model selection?
> 2- it has happened to me that, while the full model converged, the reduced
> model gave a convergence warning message. Considering that my only goal was
> to test whether the effect of treatment was significant or not, how would
> the convergence issue influence my conclusions?

> I feel like these are very basic questions but very important because I do
> not want to draw wrong conclusions. Thanks,

Personally, I think these are kind of cutting edge type questions. I would looking at
diagnostics from even simpler models first (say, poisson and quasi-poisson GLM, or hurdle model), and
the same model using a different package (say in STAN with uninformative priors).