Hi Peter,
I totaly agree with Jeff and you; the different packages should be reused as
much as possible. But it requires considerable constraints on the design of
the package to be reusable by other packages with a context. The package
should be designed with reuse as an important objective in mind. A lot of
the packages I looked at for running a backtest for a trade system are
obviously not designed with this in mind. They are more geared towards a
specific solution within a rather specific context...

I think reusability of software is one of the more devine trophies of
software engineering, and is certainly not to be achieved easily. I think
standards and a framework can make a big difference here. It allows the
developer of a package to focus on the problem at hand and it provides
interfaces to glue it all together.

That all said, I got an error when I tried to run the turtle demo:

-------------------------------------- start trace
---------------------------------------------------------------
> demo(turtles)


        demo(turtles)
        ---- ~~~~~~~

Type  <Return>   to start :

> updateStrat <- function(Portfolio, Symbol, TxnDate, PosUnitsQty, UnitSize,
StopPrice, TxnPrice, TxnN)
+ { # @author Peter Carl
+
+     # DESCRIPTION:
+     # Adds transactions-related data to the STRATEGY timeseries.
+
+     # Inputs
+     # TxnD .... [TRUNCATED]

> # - Turtle System #1
>
> # required libraries
> require(quantmod)

> require(TTR)

> require(blotter)

> # Set initial values
> initDate="2005-01-01"

> initEq=100000

> print("Initializing portfolio and account structure")
[1] "Initializing portfolio and account structure"

> # Assemble a small portfolio of three stocks
> symbols = c("XLY", "XLP", "XLE", "XLF", "XLV", "XLI", "XLB", "XLK", "XLU")

> getSymbols(symbols, from=initDate, source="yahoo")
pausing 1 second between requests for more than 5 symbols
pausing 1 second between requests for more than 5 symbols
pausing 1 second between requests for more than 5 symbols
pausing 1 second between requests for more than 5 symbols
pausing 1 second between requests for more than 5 symbols
[1] "XLY" "XLP" "XLE" "XLF" "XLV" "XLI" "XLB" "XLK" "XLU"

> # Set up a portfolio object and an account object
> portfolio = initPortf(symbols)

> account = initAcct(portfolios="portfolio", initDate=initDate)

> # This table stores transaction-related information relative to the
strategy
> # Placing it into the portfolio object, sure why not?
> for(symbol in symbols){
+   portfolio[[symbol]]$strat <- xts( as.matrix(t(c(0,0,0,0,0))),
order.by=as.POSIXct(ini
.... [TRUNCATED]

> # @todo: decrease the size of the notional account by 20% each time lose
10% of original account (10% drawdown).  E.g., if trading a $1M account and
down 100K, trade as if $800K account until out of drawdown.  If lose another
10% from 800K, or 80K  .... [TRUNCATED]
[1] "Setting up indicators"

> for(symbol in symbols){
+     # System 1
+     #
+     # 20-day breakouts are ignored if the last breakout
+     # would have resulted in a winning trade
+     #
+     # These values will also be used as System 2 exits
+     x=get(symbol)
+     # E .... [TRUNCATED]

> # Portfolio Parameters
> size = 0.01

> maxUnits = 4

> Units=0

> verbose=TRUE

> # Create trades
> for( i in 57:NROW(x) ) { # Assumes all dates are the same
+   CurrentDate=time(x)[i]
+   print(CurrentDate)
+   equity = getEndEq(account, CurrentDate)
+
+   for(symbol in symbols){
+ #      print(symbol)
+     x=get(symbol)
+    .... [TRUNCATED]
[1] "2005-03-24"
[1] "XLY 2005-03-24 -94 @ 32.92"
[1] "XLP 2005-03-24 -200 @ 22.91"
[1] "XLF 2005-03-24 -93 @ 28.1"
[1] "XLK 2005-03-24 -184 @ 19.37"
Error in updatePortf(Portfolio = portfolio, StartDate = CurrentDate, EndDate
= CurrentDate) :
  unused argument(s) (StartDate = 12866, EndDate = 12866)
>
--------------------------------------------- end trace
-------------------------------------------------------------------

I will definitly have a second look at blotter again; maybe I gave up on it
to soon. I will probably ask some questions on the list about it, as I
remember encountering some issues trying to use it as a blotter with the
context of backtesting a single instrument in mind.

Regards,

-Mark-

2009/7/17 Peter Carl <peter@braverock.com>

> I'll add to Jeff's comments a little, but I think he's absolutely right
> about working towards integration of various components.
>
> PerformanceAnalytics has always been intended for statistical evaluation
> of the resulting returns generated by trading systems, and has been
> carefully scoped that way.  Although we started writing it before zoo or
> xts came along, we had long wished for a time series package such that we
> could concentrate on the objectives of the package rather worry about
> underlying mechanics.  For me, zoo and xts are key components not only
> because they make development easier, but because they should also make
> integration easier.  That's why we're continuing to work to  move away
> from our own (legacy?) code and take advantage of all that xts offers.
>
> Blotter is another example of a component.  blotter takes it's name from
> the "trade blotter," which is used by traders for recording transactions.
> blotter is *not* about creating trading strategies, but only about
> managing trades, positions, and calculating p&l on those trades/positions.
>  We will be extending it shortly to handle multiple instrument types and
> currencies, as our requirements are for worldwide markets and multiple
> different asset classes.  So, in that paradigm, the "trading system" would
> generate a list of trades, and 'blotter' would be an interface for
> evaluating performance, keeping records, etc., completely agnostic of the
> process that generates trades.
>
> Although blotter is still immature code, it's pretty functional already
> given that it benefits so much from the existence of xts.  To see how to
> use blotter, install it from r-forge and take a look at:
> library(blotter)
> example(blotter)
> demo(turtles)
>
> The example is a relatively simple one with a few trades in a few stocks.
> The demo is a more developed example of a trading system.  In this case,
> it's a trend following system originally developed by Josh Ulrich that
> uses signals from TTR, charts and prices from quantmod, and blotter to
> calculate P&L.  There's a lot to do still, but I think it hints at the
> possibilities here.
>
> pcc
> --
> Peter Carl
> http://www.braverock.com/~peter
>
> > All,
> >
> > I suspect this topic/conversation is central to most of us -- and
> > quite important to R and R-Finance in general.
> >
> > There are two primary issues that we seem to be encountering at this
> > point in the R-Finance community:
> >
> > 1) consistency of design: so all of our work becomes additive.
> > 2) generality: writing code that works in the most general case possible
> >
> > The first is quite difficult.  I have personally been involved in
> > *hundreds* of hours of productive (?) conversation with dozens of
> > people on this topic, be it on the list, useR, NY, Meielisalp,
> > R/Finance Chicago and back again at Meielisalp (and many sessions
> > involving Jak's tap in Chicago).  All of the conversations have been
> > certainly valuable, though even after all of this I still can't figure
> > out the best path...
> >
> > One thing that I have worked on personally is making consistency
> > possible.  'xts' was born of this issue.  We didn't need another
> > time-series class.  We had 9 or 10.  We needed (or I needed) a way to
> > abstract the class and just get the job done.  'xts' does that
> > reasonably well -- though still far from complete.  The same issue
> > exists for portfolio level data, as well numerous other 'classes'.
> >
> > Everyone should be free to use their own data object, what would be
> > nice (and required for maximum usefulness in my opinion) is if that
> > choice wasn't imposed on the end-user to use your software.  This
> > leads to more users, more feedback, better code, and a more 'complete'
> > landscape of tools that can be used.
> >
> > As a time series example -- xts lets the developer use one type of
> > object (simplifying code), and accept *ALL* objects (simplifying the
> > user experience).
> >
> > The point of the above is that interface counts, and it counts for a
> > lot.  If a framework is to work, it needs to be accessible to all
> > users.
> >
> > The second point I'll toss out there is one of generality.
> >
> > This is just as difficult as the first conceptually, though I would
> > argue possibly even more intractable.  We just can't individually
> > understand what we collectively require.
> >
> > quantmod set about in 2007 trying to create a 'framework' and has
> > failed miserably.  It does some things like data management and
> > charting quite well, but the often asked question is how does
> > specifyModel/buildModel/tradeModel work.  The short answer is that it
> > does, and it doesn't.  It works on one type of strategy (EOD), and
> > doesn't on others:
> >
> >> ?tradeModel
> >>      ## Not run:
> >>      m <- specifyModel(Next(OpCl(QQQQ)) ~ Lag(OpHi(QQQQ)))
> >>      m.built <-
> >> buildModel(m,method='rpart',training.per=c('2007-01-01','2007-04-01'))
> > loading required package: rpart
> >>
> >>      tradeModel(m.built)
> >
> >   Model:  rpart1247761259.05370
> >
> >   C.A.G.R.:  16.96%     H.P.R.:  67.94%
> >
> >   Returns by period summary:
> >
> >             weekly monthly quarterly yearly
> >     Max.    12.29%  18.96%    21.12% 52.23%
> >     3rd Qu.  2.43%   7.51%     8.84% 28.51%
> >     Mean     0.49%   2.10%     5.46% 20.50%
> >     Median   0.47%   0.78%     4.58%  4.80%
> >     2rd Qu. -1.63%  -3.01%     0.80%  4.63%
> >     Min.    -8.89% -10.71%    -4.87%  4.46%
> >
> >   Period to date returns:
> >
> >              weekly monthly quarterly yearly
> >              -0.44%   0.75%     0.75%  4.46%
> >>      tradeModel(m.built,leverage=2)
> >
> >   Model:  rpart1247761259.05370
> >
> >   C.A.G.R.:  29.76%     H.P.R.:  136.82%
> >
> >   Returns by period summary:
> >
> >              weekly monthly quarterly  yearly
> >     Max.     25.22%  36.09%    36.23% 105.40%
> >     3rd Qu.   4.72%  14.80%    17.59%  56.63%
> >     Mean      0.97%   4.05%     9.56%  39.50%
> >     Median    0.87%   1.46%     7.90%   7.90%
> >     2rd Qu.  -3.38%  -6.20%     1.34%   6.57%
> >     Min.    -17.15% -20.97%   -10.30%   5.24%
> >
> >   Period to date returns:
> >
> >              weekly monthly quarterly yearly
> >              -0.95%   1.35%     1.35%  5.24%
> >>      ## End(Not run)
> >
> > Looks great!  But it is useless.  It is a casualty of specificity.  A
> > generalized framework would be awesome, but I think it is a much
> > larger task than I can handle.  I suspect that general to one person
> > is fantastically restrictive to another.
> >
> > R is the 'general' framework.
> >
> > What we really need is consistency of the pieces that make
> > building/testing models in *R* easier.
> >
> > This comes from projects like blotter I think.  Were we can use parts
> > that we want, and only those that we want.
> >
> > As I said at the start, this is a lot more involved than it looks
> > like.  Obviously best of luck to all who take up the challenge.  This
> > thread is an awesome start to the (larger) public conversation that
> > has been taking place over beers for quite some time now.
> >
> > Best,
> > Jeff
> >
> > On Thu, Jul 16, 2009 at 9:41 AM, Robert Sams<robert@sanctumfi.com>
> wrote:
> >> Hi Mark,
> >>
> >> I have started a package called tradesys which, I think, is a clean
> >> solution. The project is registered on r-forge
> >> https://r-forge.r-project.org/projects/tradesys/ and the initial code
> >> base with documentation will be checked-in by the end of the day London
> >> time. As a taster:
> >>
> >>> library(tradesys)
> >>> data(spx)
> >>> tail(spx)
> >>             Open   High    Low  Close     Volume
> >> 2009-05-12 910.52 915.57 896.46 908.35 6871750400
> >> 2009-05-13 905.40 905.40 882.80 883.92 7091820000
> >> 2009-05-14 884.24 898.36 882.52 893.07 6134870000
> >> 2009-05-15 892.76 896.97 878.94 882.88 5439720000
> >> 2009-05-18 886.07 910.00 886.07 909.71 5702150000
> >> 2009-05-19 909.67 916.39 905.22 908.13 6616270000
> >>> x <- tradesys(spx, el=MA(Close, 60) > MA(Close, 120), es=MA(Close, 60)
> >> <= MA(Close, 120))
> >>> tail(trades(x, uselog=TRUE))
> >>    phase      etime      xtime time nobs  eprice  xprice    pnl
> >> ror
> >> 107    EL 2006-09-21 2007-09-12  356  244 1324.89 1471.10 146.21
> >> 0.264117052
> >> 108    ES 2007-09-12 2007-11-09   58   42 1471.10 1467.59   3.51
> >> 0.006027153
> >> 109    EL 2007-11-09 2008-01-03   55   36 1467.59 1447.55 -20.04
> >> -0.034689959
> >> 110    ES 2008-01-03 2008-06-10  159  109 1447.55 1358.98  88.57
> >> 0.159301490
> >> 111    EL 2008-06-10 2008-07-21   41   28 1358.98 1261.82 -97.16
> >> -0.187159273
> >> 112    ES 2008-07-21 2009-05-19  302  209 1261.82  909.67 352.15
> >> 0.825619186
> >>> tail(equity(x, uselog=TRUE))
> >>           trade states    delta    price          ror   equity
> >> 2009-05-12   112     -1 1.331220 6.814016  0.018107970 38.02601
> >> 2009-05-13   112     -1 1.307543 6.808377  0.007373275 38.30638
> >> 2009-05-14   112     -1 1.297973 6.784729  0.030694874 39.48219
> >> 2009-05-15   112     -1 1.259318 6.794318 -0.012075942 39.00541
> >> 2009-05-18   112     -1 1.274712 6.786796  0.009588169 39.37940
> >> 2009-05-19   112     -1 1.262606 6.813082 -0.033188777 38.07244
> >>
> >> Please checkout and play with the code at your leisure. Anyone
> >> interested in write-access to the repository should contact me directly.
> >>
> >>
> >> Robert
> >>
> >>
> >> ________________________________
> >>
> >>        From: r-sig-finance-bounces@stat.math.ethz.ch
> >> [mailto:r-sig-finance-bounces@stat.math.ethz.ch] On Behalf Of Mark
> >> Breman
> >>        Sent: 16 July 2009 15:15
> >>        To: r-sig-finance@stat.math.ethz.ch
> >>        Subject: [R-SIG-Finance] Backtesting trade systems
> >>
> >>
> >>        Hello,
> >>
> >>        I have spend quit some time now looking for a package that
> >> allows me to backtest (technical) trading systems based on single
> >> financial instruments with R.
> >>
> >>        I had a look at Rmetrics, blotter, fTrading,
> >> PerformanceAnalytics, backtest, quantmod, TTR etc, but not one of these
> >> fill my requirements. It's not that they are not usefull, on the
> >> contrary, they are all filled with terrific statistical stuff, but it's
> >> not the simple, practical and straightforward approach that I am looking
> >> for as a trader rather than as a statisticus.
> >>
> >>        So I have decided to build my own solution, reusing as much as
> >> possible from these existing packages. (As a former software engineer I
> >> know how much time and effort goes into buiding reliable software, so
> >> the more reuse the better). As I am quite new to R and statistics in
> >> general, there is a lot to learn for me here...
> >>
> >>
> >>        What I have build so far is a very basic set of functions called
> >> "tradesim.R" (I have attached it to this post). A very basic example of
> >> how these functions can be used for a backtest-run can be found in
> >> "tradesim_example.R". The example runs a backtest with end-of-day data
> >> from AAPL, using a (rather poor) trading system based on the RSI
> >> indicator (from the TTR package).
> >>
> >>        Now I have read in some older post on this list that others were
> >> also searching for a backtesting package. I even read a post proposing
> >> to start a group effort creating such a package. I suspect that some of
> >> you might be interested in what I made so far and maybe would like to
> >> put in a effort creating such a package together. I certainly know that
> >> it's a lot easier to create good software as a group, rather than by a
> >> single person...
> >>
> >>        So if you are interested have a look at what I got so far and
> >> let me know what you think.
> >>
> >>        Regards,
> >>
> >>        -Mark-
> >>
> >>
> >>
> >> _______________________________________________
> >> R-SIG-Finance@stat.math.ethz.ch mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >> -- Subscriber-posting only.
> >> -- If you want to post, subscribe first.
> >>
> >
> >
> >
> > --
> > Jeffrey Ryan
> > jeffrey.ryan@insightalgo.com
> >
> > ia: insight algorithmics
> > www.insightalgo.com
> >
> > _______________________________________________
> > R-SIG-Finance@stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only.
> > -- If you want to post, subscribe first.
> >
>
> _______________________________________________
> R-SIG-Finance@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>

	[[alternative HTML version deleted]]

