[R-SIG-Finance] exponentially weighted linear regression

[Eric Zivot]

It seems like this can be easily solved using rollApply or period.apply and
lm with exponentially declining weights (in the w option). BTW, regression
on exponentially weighted data is often called "discounted least squares"
and is what I would call a "poor man's" kalman filter. In fact there have
been a few papers in the engineering literature which shows that discounted
least squares is equivalent to a certain type of filtered estimated from a
state space regression model with time varying parameters. I've used
discounted least squares for modeling hedge fund data and found it to work
quite nicely. Also, the time varying parameter state-space model may work
well with a few explanatory variables (1 or 2) but it often does not perform
well if there are many (say 5+ predictors). For each regression parameter,
you have to estimate an AR(1) smoothing parameter plus a transition equation
error term. With many predictors, you run into numerical stability problems
very quickly and the likelihood function can have many local minima. For
this reason, discounted least squares is an attractive alternative. Finally,
it is very difficult to do model selection with the state space kalman
filter. How do you choose the variables to enter the regression equation? I
have not seen anyone do a systematic study of model selection in time
varying parameter models. Perhaps, this is where a Bayesian approach might
be useful. This is an important but neglected topic. I would be happy is
someone pointed me to some research on this topic.


Eric Zivot                  			               
Robert Richards Chaired Professor of Economics and Director of Outreach
Adjunct Professor of Finance                            
Adjunct Professor of Statistics
Adjunct Professor of Applied Mathematics 
Department of Economics
Box 353330                  email:  ezivot at u.washington.edu 
University of Washington    phone:  206-543-6715            
Seattle, WA 98195-3330
www:  http://faculty.washington.edu/ezivot                  




-----Original Message-----
From: r-sig-finance-bounces at r-project.org
[mailto:r-sig-finance-bounces at r-project.org] On Behalf Of riccardo visca
Sent: Thursday, January 05, 2012 7:44 AM
To: Zachary Mayer; Michael
Cc: r-sig-finance
Subject: Re: [R-SIG-Finance] exponentially weighted linear regression

I think you should use a time varying parameter model (look at dlm package)
or if you do not have time to learn try to smooth the predictors first then
run the rolling window regression maybe robustified.

If you use the Kalman filter use an AR(1) for the parameter in the state
equation like

yt = bt * Xt-1
bt = c * bt-1

c will do the smoothing (it is your (1-lambda))

>From my point of view there is no cheap and dirty solution to this problem.




________________________________
 Da: Zachary Mayer <zach.mayer at gmail.com>
A: Michael <comtech.usa at gmail.com>
Cc: r-sig-finance <r-sig-finance at stat.math.ethz.ch>
Inviato: Giovedl 5 Gennaio 2012 15:22
Oggetto: Re: [R-SIG-Finance] exponentially weighted linear regression

I'm not following you.  What do you mean by "a divided trunk of data?"  As
far as I know, lm and glm will take as much data as you give them, until you
run out of memory.  Why not run the lm on the whole time series?

Perhaps you could post a reproducible example, so we can see what you're
trying to do.

On Thu, Jan 5, 2012 at 10:16 AM, Michael <comtech.usa at gmail.com> wrote:

>
> Thanks Zach.
>
> The problem with these functions (e.g. lm or glm with weights 
> argument) is that they still do it on a divided trunk of data... i.e. 
> block by block, not the whole time series...
>
> If we think about the exponential moving average estimate of 
> volatility with a decay factor lamda, it is actually on the whole time 
> series... not divided trunk of data.
> Any thoughts?
>
> On Thu, Jan 5, 2012 at 8:40 AM, Zachary Mayer <zach.mayer at gmail.com>wrote:
>
>> Hi Michael,
>>
>> R has lots of functions for exponential smoothing.  
>> ets<http://rgm2.lab.nig.ac.jp/RGM2/func.php?rd_id=forecast:forecast.e
>> ts> in the forecast 
>> package<http://cran.r-project.org/web/packages/forecast/index.html> 
>> is particularly useful.  Additionally, many models in R (including lm and
glm) have a weights argument.  You can come up with any weighting scheme you
wish, and pass it to your model as a weights argument.
>>
>> There may be more useful references on the CRAN time-series task 
>> view<http://cran.r-project.org/web/views/TimeSeries.html>
>> .
>>
>> Regards,
>>
>> Zach
>>
>>  On Thu, Jan 5, 2012 at 9:30 AM, Michael <comtech.usa at gmail.com> wrote:
>>
[[elided Yahoo spam]]
>>>
>>> Is there a function for exponentially weighted linear regression in R?
>>>
>>> Usually, a linear regression is on a trunk of data...
>>>
>>> And if I run linear regression on time series, I divide the time 
>>> series into "overlapped/rolling" windows and run linear regression 
>>> on each rolling chunk of data...
>>>
>>> Is there a way to turn the rolling linear regression on the whole 
>>> time series into an exponentially weighted one, i.e. using a decay 
>>> factor lambda to give more weights to the newer observations...?
>>>
>>> Are there packages in R which can do that?
>>>
>>> Thanks a lot!
>>>
>>>        [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-SIG-Finance at r-project.org mailing list 
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R 
>>> questions should go.
>>>
>>
>>
>

    [[alternative HTML version deleted]]

_______________________________________________
R-SIG-Finance at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions
should go.
	[[alternative HTML version deleted]]