[R-SIG-Finance] AIC and deeper insight into model comparison

[alexios ghalalanos]

Hi Gareth,

On 04/09/2014 08:54, Gareth McEwan wrote:
> Dear all
> 
> Using reference paper "Ng, S. and Perron, P. (2005). A Note on the
> Selection of Time Series Models. *Oxford Bulletin of Economics and *
> *Statistics*, 67 (1):115-134."
> 
> In fitting ARMA(p,q)-GARCH(1,1), ARMA(p,q)-GJR-GARCH(1,1) and
> ARMA(p,q)-EGARCH(1,1) models to a time series (p,q = 0,1,2) under the
> available error specifications (norm, std, ged, ghyp, sstd, sged, jsu,
> nig), I had in mind to choose the fit with the lowest AIC value.
> 
> The Ng and Perron paper, however, advise holding N (the number of
> observations) *fixed* (bottom of p.8) if you want theoretically valid model
> comparisons. I assume R fits all the data in ARMA(0,q) and would drop 1
> observation in fitting ARMA(1,q) and, similarly, would drop 2 observations
> in fitting ARMA(2,q).
That depends on the implementation. Some implementations try to build up
the "model" as data is made available rather than dropping everything
upto the highest lag order i.e for an AR(2) model, at time T=1 use the
unconditional mean (if that is included in the model), at time T=2 apply
the ar1 coefficient, and at T=3 the ar2 coefficient.
> 
> Has anyone dealt with this model selection issue before? If so, could you
> recommend an approach that would allow N to be fixed for all fittings.
> Note: I think the way to address the problem is to use the whole dataset
> for ARMA(2,q), then drop the first observation when fitting ARMA(1,q), and
> drop the first 2 observations when fitting ARMA(2,q)...thinking that this
> would use R in a way that leads to theoretically correct AIC values valid
> in comparing models.
In rugarch the information criteria are scaled by the number of
observations N e.g. the AIC is calculated as:
>(-2*LLH)/N + 2 * m/N
where m is the number of parameters.

This therefore adjusts the statistic to yield a per observation
contribution.

Whether this is a "theoretically" valid way or not to compare these
compare models I cannot say for sure.
Unless your dataset is very small, includes very large outliers in the
first "q" observations or you are using a very large number of lags,
how important do you think will be a couple of points in the overall
comparison?
> 
> Any help or guidance would be appreciated.
> 
> Many thanks
> Gareth McEwan
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-SIG-Finance at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
> 

Alexios