[R-SIG-Finance] Stability of trading models

[Gero Schwenk]

Hello together!
I've got a question regarding the assessment of stability of trading 
models. Here's a short intro to the background:

I have fitted regression models with lagged independent variables to 
data of daily stock returns. (The model methodology was MARS, allowing 
for  second order interaction terms.) The independent variables had been 
selected on basis of explorative clustering and correlation studies and 
showed face validity. I used both logit and gaussian link functions and 
generated out-of-sample predictions for a test window of two months (40+ 
observations, following the training window in time), which were of 
reasonable to really satisfying quality, depending on the exact model.

In order to assess the stability of out-of-sample fit of a given model, 
I would normally draw cross-validation samples and partition them into 
training- and test subsets. Grounds for this would be the assumption of 
independent observations contained in the model and forced onto the data 
by backshifting them.

However, I'm reluctant to believe that the data-generating process 
doesn't change over time, which is implied by my procedures. If this was 
true and time was not an issue, it should not be necessary to 
recalibrate the model, even after a long period of out-of-sample 
prediction. This seems overly optimistic to me.

Returning to the question of stability assessment and cross-validation, 
I would like to know if there is some pragmatic solution. Is simple 
cross-validation viable? Do I need to go far into the past using some 
possibly sliding training- and test-windows? Or has anybody a different 
suggestion how to deal with this problem in the realm of regression models?


Kind regards,
Gero