[R] Testing predictive power of ARIMA model

[Gad Abraham]

Evan DeCorte wrote:
> Thanks for the great feedback. Conceptually I understand how you would go about testing out of sample performance. It seems like accuracy() would be the best way to test out of forecast performance and will help to automate the construction of statistics I would have calculated on my own. 
> 
> However, the real question now is how do you loop through a time series and automatically split a time series into training and testing sets. I know how I would do it for individual sets but to do so manually over a large number of time series seems excessively burdensome. 

You don't have to do it manually. For example, if you want to do 10-fold 
cross-validation, and you have a time series of length n, then split it 
into n/10 blocks, e.g. using the index i <- rep(1:10, each=n/10) 
(assuming n is divisible by 10), loop 10 times using 9 blocks as 
training and 1 block as test (different test block each time) and 
measure the MSE for each repetition. Repeat this for all your time series.


-- 
Gad Abraham
Dept. CSSE and NICTA
The University of Melbourne
Parkville 3010, Victoria, Australia
email: gabraham at csse.unimelb.edu.au
web: http://www.csse.unimelb.edu.au/~gabraham