[R-SIG-Finance] Strategy performance summary report

[Vijay Vaidyanathan]

Hi Daniel,

On May 2, 2011, at 1:16 PM, Daniel Cegiełka wrote:

> Peter,
> 
> Of course you right - someone can test strategies using the blotter.

Well, Blotter, PerformanceAnalytics/ReturnAnalytics and other packages are key ingredients *around* a strategy simulator (of which a backtester is a special case). You still need something like quantstrat to "drive" the test.

> When I first used the blotter, I was convinced that this is how it
> should work (for backtesting). Next I thought, if I use the blotter
> backtest, it can also be used for real trading. I noticed that blotter
> is too slow and I started to look for ways to make it faster with a C.

I'm not sure what you found slow - I've played with it a little bit and considering the early state, it seemed like speed was not an issue for me. Could you post an example or some specifics?

> :
> Beginners often
> commits the same mistake that I committed - that is, believes that eg
> blotter is a whole platform. But the truth is that this is only a
> small part which is designed specifically for PnL (blotter). To
> analyze the strategy can be used with some packages from
> ReturnAnalytics - all specialized in selected uses.

Yes, you do need a separate "strategy simulator" (which I'll just call a backtester for now, although you can forward test just as easily, with simulated data). You can backtest using blotter and custom code, or quantstrat which uses a signal based strategy testing paradigm.  A few years ago, I wrote a backtesting framework for some academic research (which tends to use screening based strategies, rather than signal based strategies). While you can use screen based frameworks for signal based strategy testing, and vice-versa, as a broad sweeping generalization, academics tend to think in terms of screening/sorted portfolios, while practitioners, particularly traders, tend to think of signal-driven portfolios.

If you are interested in the screening/sorting approach, feel free to look at my screening-based backtesting framework PAST.  Its a different paradigm than signal based trading, so you need to wrap your head around that if you are used to thinking of backtesting "trading signals". The docs (such as they are) and code are here if you want to look at it (although, it is definitely slow!).

http://www.returnmetrics.com/sw/past/docs/

That page also has a link to the code, which I have been using on a fairly regular basis for about 5 years. Unfortunately, it has been stuck in a "works for me" status for the past two years :-(

The good news is that I'm currently reviewing resumes for a Research Assistant hire over the summer, and the first thing he or she will be working with me on is getting PAST to the point where it is documented and on CRAN. Next, I hope to integrate blotter and/or quantstrat either tightly as a required package, or loosely (as I do with PerformanceAnalytics now, as a recommended package). 

Unfortunately, until then. all I have is very meager documentation and a few data base drivers for the AAII database and for historical data loaded into a dataframe.

regards,

- Vijay
======
[PS: If anyone on this list knows of a good RA candidate to work on this over the summer, I'd appreciate any leads. It's a paid gig (albeit at grad-student rates) and also presents the opportunity to learn an enormous amount in a short time, while working on an open-source project and making the world a better place!]