
> 
> From: Ben Bolker <bbolker@gmail.com>
> Subject: Re: [R-sig-ME] Autocorrelation in a GAMM for nightly time series
> Date: 5 June 2013 13:35:51 NZST
> To: r-sig-mixed-models@r-project.org
> 
> 
> Andrew Digby <andrewdigby@...> writes:
> 
>> I'm analysing call counts of a nocturnal species in 
>> response to temporal and environmental effects. Over a
>> period of 3 years I divide each night into 10 equal-length blocks, 
>> and count the calls of each sex in each
>> block. 
> 
> [snip]
> 
>> gam(ncalls ~ sex + year + s(month) + s(time of night) +
>> s(moon) + s(temperature) + s(rain) +
>> offset(blocklength), family=negbin(c(1,5)), data=dc)
> 
>> Where 'time of night' = 1-10 = the block number during the night,
>> and blocklength = the length of that block (this changes throughout
>> the year; hence the offset).
> 
>> Not surprisingly, the acf shows significant correlation between
>> adjacent time blocks, with a periodicity of 10. This is because
>> counts in a particular block will be similar to those in the blocks
>> immediately before and after, and also to the same block on other
>> nights (the species has a quite regular pattern of decreasing call
>> rates as the night progresses).
> 
>> To address these autocorrelation problems, I've tried various 
>> ARMA correlations, with correlation
>> between time of night blocks, grouped by day (or week) and sex:
>> 
>> gamm( .., correlation=corARMA(form=~TimeofNight | DayofYearSex), p, q) 
>> (p=1:3, q=0:3)
>> gamm( .., correlation=corARMA(form=~TimeofNight | WeekofYearSex), p, q)
> 
>> This improves the ACF, but there are still correlations between
>> residuals at same time of night - e.g. peaks every 10 lags.
> 
>> 1) Can I rely on the ACF when my time series isn't actually
>> consecutive time bins, since it only includes each night (not full
>> days). This means, for example, that the time lag between
>> observations 9 & 10 (~1 hour) is much longer than that between 10 &
>> 11 (~12 hours = daytime)?
> 
>> 2) (if yes to the above) How can I construct a correlation structure
>> in a GAMM which allows for correlation between adjacent time bins
>> within each night, and between the same time bin on different
>> nights?
> 
>  I think you can use corCAR1, which is intended for continuous-time
> models, but it's limited -- it does only exponential correlations
> in time (analogous to corAR1). 
> 
>  If you have enough data/computational power, you might try 
> a 2D spline (tensor product? I don't quite remember) to allow
> for a gradually changing nocturnal time pattern over the course
> of the year (i.e something like s(timeofnight,dayofyear)) --
> not entirely clear to me why you're fitting s(month) rather than
> s(dayofyear), unless you only have one observation per month?
> 
> In general my experience is that when I find myself fitting
> high-order ARMA models (i.e. more than a couple of lags in total),
> it's because there's some systematic pattern that I failed to
> capture in the fixed-effects part of the model. Your mileage may
> of course vary.
> 
>  Ben Bolker
> 
> 
> 


Thanks very much for that, Ben. The amount of help you provide through these forums is truly impressive!

I've tried your three suggestions, but unfortunately they don't help with the autocorrelation. I was also convinced I must be missing a covariate, but lots of tests haven't shown what it could be. The tensor product provided some really helpful information about seasonal changes in patterns, so I'll keep that in, but it didn't affect the acf. I was fitting s(month) to compare with previous analyses, but you're right that s(dayofyear) is more appropriate for this model.

So thanks to your help the model is improved, although the autocorrelation problem remains. I have a couple of remaining questions regarding this, which I'd really appreciate advice with: 

1) ACF confidence intervals: because I have a large dataset (~40,000 counts in ~11,000 bins), the 95% confidence intervals of the acf [acf(resid(gamm.model, type='n')] are tiny: about +/- 0.01. So while the peaks in the ACF  exceed these, they're still small correlations: ~0.1. Because my data have lots of time gaps in them, since they're just lots of nights merged together, can I rely on these confidence intervals and ACF calculation? Am I overstating the importance of the ACF? When I construct a variogram (as per Zuur et al's 2009 book), it's pretty flat, suggesting that there are no major independence problems.

2) This model seems to need a complex correlation structure: an AR correlation between time bins in the same night (TimeofNight | NightofYear), AND correlation (AR?) between the same time bin on different nights (1 | TimeofNight). Several sources warn against using over-complicated correlation structures. But if the ACF shows residual autocorrelation with more basic structures, isn't a more realistic structure needed; or am I putting too much emphasis on the ACF CIs (question 1)?

Again, many thanks.

Andrew
	[[alternative HTML version deleted]]

