[R] Negative binomial GAMM using 'by' in factor interactions

[Eva Maria Leunissen]

I am using a GAMM to model my data (this is as far as I know the only way I
can use the negative binomial distribution AND a correlation structure
within the model).

I measured animal detections (including zero detections) per hour at 3
different locations in an area. location is a factor in my model and the
other possible explanatory variables are environmental variables and level
of disturbance.

I'm expecting the response to be different at the 3 different locations for
each variable so have been modelling the terms as interactions with each of
the 3 factor levels of location using the 'by' argument in the 'ti'
smoothing term, as well as 'location' as a variable by itself. Does it make
sense to include the main effect as well as the interaction term? for
example for including the variable 'windspeed': if including the term
ti(windspeed, by=location), is it necessary to also include s(windspeed)?
or would it only make sense to inlcude them in separate models only and
compare the models?

As far as I understand the 'by' argument calculates a separate smooth for
each of the factor levels, so if the effect was the same at each location
it wouldn't hurt to use the 'ti' smooth with the 'by' argument if the
effect of the variable was the same at each location.

The issue I'm having is that by including both terms and then doing model
selection gives me many very similar models within a deltaAIC of 6 of the
best model, where the differences lie in the inclusion of main effects when
the 'interaction' is also there. The inclusion of the interaction term
gives bigger changes in AIC compared to the inclusion of the main effect.

This brings me to my other question. Is it possible to compare GAMMs with a
negative binomial family using AIC? e.g. using AIC(mod$lme). If not, what
is the best way to compare them?

Thank you very much for your time

	[[alternative HTML version deleted]]