[R-SIG-Finance] Causal version of HP filter and Kernel Smoothing in R?
Wildi Marc (wlmr)
wlmr at zhaw.ch
Sun Feb 26 14:20:36 CET 2012
Being interpellated I provide cursory feedback on the topic. R-code illustrating the HP filter (symmetric or real-time) is proposed below.
Modern (traditional) filter design goes back to early work by Wiener and Kolmogorov (WK). State-space models (Kalman-filter/smoothing) is an alternative formulation - parameterization - of the problem which is found more convenient/appealing by proponents: Harvey has highlighted some of the natural appeal in economic applications. In general, both approaches could replicate each other by suitable reformulation (parameterization) of models. So ultimately it is a matter of taste which is to be preferred. I frequently use state-space for didactical purposes (not that the filter is `simpler'; but `outputs' are appealing to non-experts).
HP-filter goes back to ideas proposed by Whittaker: it is ultimately a tradeoff between `fit' and `smoothness'. One can derive an ARIMA-model which replicates HP perfectly (it is a model with a double unit-root in frequency zero).
All these approaches are pure mean-square incarnations: real-time filter minimize the mean-square error between concurrent and final (symmetric) filters assuming that the (implicit or explicit) data-model is `true'.
My filter-design emphasizes a more general `customized' perspective: one can replicate ordinary mean-square designs (WK, Kalman, HP). But one can also emphasize alternative research priorities such as `timeliness', `noise suppression', `accuracy'. Depending on your priorities these aspects might be more relevant - to you -. customization means that the user can tweak the optimization criterion in order to match his individual priorities. My experience is that practitioners frequently assign priorities (in forecasting) differently than assumed by the `mean-square' paradigm.
HP-filter can be implemented in R as follows:
library(mFilter)
# articial data: white noise (not very clever because HP assumes a double unit-root)
set.seed(1)
len<-201
eps<-rnorm(len)
# lambda=1600 is a typical setting for working with quarterly GDP
lambda_hp<-1600
eps.hp <- hpfilter(eps,type="lambda", freq=lambda_hp)
# plot data (here noise) and HP-trend
plot(eps.hp$x)
lines(eps.hp$trend,col=2)
# Here is the coefficient matrix: it is a full revision sequence.
parm<-eps.hp$fmatrix-diag(rep(1,len)
parm<--parm
# And a plot of the HP-filter coefficients: symmetric and real-time (concurrent) trend filters
ts.plot(cbind(parm[,c(length(parm[1,])/2,length(parm[1,]))]),lty=1:2)
Marc
________________________________
Von: Mark Leeds [markleeds2 at gmail.com]
Gesendet: Samstag, 25. Februar 2012 18:25
Bis: Michael
Cc: Wildi Marc (wlmr); r-sig-finance at r-project.org
Betreff: Re: [R-SIG-Finance] Causal version of HP filter and Kernel Smoothing in R?
Hi: I'm not familar with Prof. Wildi's filters or the HP filter but the standard kalman
filtering/smoothing algorithms are in dse, sspir and dlm. Well, definitely dlm has
smoothing but I bet sspir and dse have them also.
I'm most familiar with the notation in dlm so I've only used that
package. note that the kalman filtering notation can be extremely confusing because the parameter symbols change from one KF setup to another and even the time subscripting notation can vary depending on whether t is shifted in the observation equation or not. but, if you're not used to one notation, then any of them can probably help you to some degree. paul didn't mention dse probably because he didn't want to self-promote !!!!!! It's also appreciated if Prof Wildi wants to comment because maybe
his filters are very different from the ones in the packages I mentioned and I could
be leading you astray.
mark
On Sat, Feb 25, 2012 at 12:09 PM, Michael <comtech.usa at gmail.com<mailto:comtech.usa at gmail.com>> wrote:
Using you guys' term, I suspect the paper used "2-sided" filter, which
"peeks" into the future, so the result in that paper is bogus...
I want to find the "1-sided" version of HP filter and Kernel Smoother, and
see if they work...
Any thoughts and pointers?
Thanks!
On Sat, Feb 25, 2012 at 1:48 AM, Wildi Marc (wlmr) <wlmr at zhaw.ch<mailto:wlmr at zhaw.ch>> wrote:
> I'm using the term as an `economist' and/or as an `engineer', hopefully
> without adding confusion to the topic. Specifically, a real-time filter is
> one-sided (which might sound redundant to some): it is also called
> concurrent filter in time series analysis (where target signals are
> considered to be outputs of bi-infinite symmetric filters: not
> smoothers...). Real-time data means: data as it arrives (possibly being
> revised in later vintages). A paper constructing real-time filters in the
> case of real-time data (mixing both concepts) is proposed in
> http://blog.zhaw.ch/idp/sefblog/index.php?/archives/205-7th-Annual-CIRANO-Workshop-on-Data-Revision-in-Macroeconomic-Forecasting-and-Policy.html
>
> Marc
>
> ________________________________________
> Von: r-sig-finance-bounces at r-project.org<mailto:r-sig-finance-bounces at r-project.org> [
> r-sig-finance-bounces at r-project.org<mailto:r-sig-finance-bounces at r-project.org>]" im Auftrag von "Paul
> Gilbert [pgilbert902 at gmail.com<mailto:pgilbert902 at gmail.com>]
> Gesendet: Samstag, 25. Februar 2012 01:04
> Bis: r-sig-finance at r-project.org<mailto:r-sig-finance at r-project.org>
> Betreff: Re: [R-SIG-Finance] Causal version of HP filter and Kernel
> Smoothing in R?
>
> Some pedantic points regarding correct terminology:
>
> On 12-02-24 06:00 PM, Brian G. Peterson wrote:
> > As usual, it helps to use the correct terminology.
> >
> >
> > The term usually employed is not 'causal' but 'one sided' or 'two sided'
> > filters.
>
> Economists usually employ the terms 'one sided' and 'two sided'. In
> engineering, physics, and mathematics, I think the terms 'filter' and
> 'smoother' are still used. (But yes, 'causal' usually has to do with
> something else.)
>
> > In classic state space models, the two sided filter is often
> > called a 'smoother', and the one-sided version is called a 'filter'.
> > See any introduction to Kalman filters for examples, since the Kalman
> > may easily by one sided or two sided.
>
> Even in the classic case this is not specific to state-space models. The
> term filter meant it could be used to filter incoming signals without
> knowledge of the future, while a smoother needs future information. So,
> a filter could be used to do realtime control, while a smoother could not.
> >
> > High pass filters are also quite trivial, as equation 4 in your
> > reference demonstrates.
> >
> > I may be incorrect, having spent only a few moments on it, but I see
> > nothing in this paper to indicate that the kernel smoothing in equation
> > 6 is not equally trivial.
> >
> > Marc Wildi has written extensively on the topic of real time (one-sided)
> > filters, and his R code is public.
> >
> Engineers use the term 'realtime data' to mean what I think most people
> would understand as 'look at the data as it is arriving', which implies
> using a (one-sided) filter. Economists use the term 'realtime data' to
> mean 'look at the data as it arrived'. That is, the vintages of the data
> that were available at different points in time. Thus a realtime
> analysis for an economist is a consideration of the revisions in the
> data. I think Marc Wildi uses the term as an economist, not as an engineer.
>
> (I warned you this is pedantic.)
> Paul
>
> > On Fri, 2012-02-24 at 16:47 -0600, Michael wrote:
> >> Thanks Brian.
> >>
> >> http://xfi.exeter.ac.uk/workingpapers/0804.pdf
> >>
> >> My understanding is that those kernel smoothers and HP filters are all
> >> non-causal...
> >>
> >> i.e. they peek into future from time-series point-of-view...
> >>
> >> Therefore, I am looking for the Causal version.
> >>
> >> Thank you!
> >>
> >>
> >> On Fri, Feb 24, 2012 at 4:41 PM, Brian G. Peterson
> >> <brian at braverock.com<mailto:brian at braverock.com>> wrote:
> >>
> >> On Fri, 2012-02-24 at 15:39 -0600, Michael wrote:
> >> > Hi all,
> >> >
> >> > I am reading a paper talking about extracting low frequency
> >> trend in FX
> >> > markets and then devising trading strategies based on those
> >> low frequency
> >> > trends.
> >> >
> >> > I was wondering if there are Causal version of HP filter and
> >> kernel
> >> > Smoothing functions in R, as mentioned in that paper?
> >> >
> >> > I did quite some search but couldn't find any ... Could you
> >> please help me?
> >>
> >>
> >> It would be easier for people to decide whether to help you if
> >> you
> >> actually provided the reference to the paper you are looking
> >> to
> >> replicate.
> >>
> >> There are many kernel smoothing methods in various R packages,
> >> which
> >> your 'quite some search' I am sure uncovered, *and* kernel
> >> smoothing
> >> mechanisms are typically rather trivial to code. So without
> >> the
> >> reference it is hard to even begin to evaluate which of them
> >> might do
> >> what you are looking for. Also, it would be polite for you to
> >> indicate
> >> in what way the kernel smoothing mechanisms provided by
> >> specific
> >> packages do not match the methodology you desire.
> >>
> >> --
> >> Brian G. Peterson
> >> http://braverock.com/brian/
> >> Ph: 773-459-4973<tel:773-459-4973>
> >> IM: bgpbraverock
> >>
> >>
> >
>
> _______________________________________________
> R-SIG-Finance at r-project.org<mailto:R-SIG-Finance at r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
> _______________________________________________
> R-SIG-Finance at r-project.org<mailto:R-SIG-Finance at r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>
[[alternative HTML version deleted]]
_______________________________________________
R-SIG-Finance at r-project.org<mailto:R-SIG-Finance at r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
More information about the R-SIG-Finance
mailing list