[R-SIG-Finance] Processing time of backtests on a single computer
Brian G. Peterson
brian at braverock.com
Wed Apr 6 22:48:47 CEST 2016
You didn't say that you were doing parameter optimization.
The length of time that a parameter optimization using brute force will
take is a linear combination of the number of parameter combinations
that you choose to search.
Typically, you should only include parameter distributions for
parameters for which you feel you have a strong economic justification.
Your strategy contains eight parameter distributions and two
constraints. The choices of these distributions appears arbitrary, and
will result in hundreds of combinations to test. So, you should expect
your strategy to take approximately a linear combination of additional
time based on the number of parameters you wish to test.
Another thing that will make this test take a long time is the inclusion
of trailing stops. As is described in the documentation, trailing stops
require evaluating the strategy at more points in the path dependent
loop. The number of observations that need to be evaluated in the path
dependent rules loop has another linear effect on the time to evaluate
your backtest. Have you validated that there is a theoretical
justification for a trailing stop? Does this increase the positive
expectation of your resulting signal process?
Most of the validation of your indicator and signal processes should be
possible long before you get to parameter optimization.
Regards,
Brian
--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock
On Wed, 2016-04-06 at 23:20 +0300, Jersey Fanatic wrote:
> The email with the CSV data attached is waiting for moderator's approval.
> The reproducible code is below.
>
>
> 2016-04-06 23:17 GMT+03:00 Jersey Fanatic <jerseyfanatic1 at gmail.com>:
>
> > Thanks for the quick reply. I did not understand what you meant by path
> > dependent looping or copying, but my aim is to decrease the processing time
> > of the backtesting of several combinations of parameters.
> >
> > Here is the reproducible example, and the data I am using is attached as
> > well:
> >
> > library(lattice)
> > library(foreach)
> > library(doSNOW)
> > library(ggplot2)
> > library(PerformanceAnalytics)
> > library(doSNOW)
> > require(latticeExtra)
> > require(grid)
> > library(gridExtra)
> > library(reshape)
> > library(quantstrat)
> > no.cores <- 8
> >
> > .strategy<- new.env()
> > .blotter<- new.env()
> >
> > currency(c('USD', 'EUR'))
> > exchange_rate(primary_id="EURUSD", tick_size=0.0001)
> >
> > data.location.r <- "processingtime_q_rsigfinance.csv"
> > symbol.data <- as.xts(read.zoo(data.location.r, sep=',',
> > tz="",header=TRUE, format='%d/%m/%Y %H:%M', index.column = 1))
> > symbol.data <- symbol.data[symbol.data$VOLUME!=0,]
> > # Delete rows with no volume (when the
> > market is closed)
> > symbol.data[,c(1,2,3,4)] <-
> > round(as.numeric(symbol.data[,c(1,2,3,4)]),abs(log10(0.0001))) #datayı tick
> > sizela uyumlu yapma
> > assign("EURUSD", symbol.data)
> >
> > strategy.st <- "rsigfinance"
> > rm.strat(strategy.st)
> >
> >
> > initDate = as.character(as.Date(index(symbol.data[1])-1))
> >
> > initPortf(strategy.st, "EURUSD", initDate=initDate, currency = "USD")
> > initAcct(strategy.st, portfolios=strategy.st, initDate=initDate,
> > initEq=100000, currency = "USD")
> > initOrders(portfolio=strategy.st,initDate=initDate)
> >
> > strategy(strategy.st,store=TRUE)
> > summary(getStrategy(strategy.st))
> >
> >
> > macdFastMARange <- seq(2,17,by=5)
> > macdSlowMARange <- seq(5,35,by=10)
> > macdSignalRange <- seq(2,18,by=8)
> >
> > StopLossDistanceRange <- seq(0.01,0.02,by=0.01)
> >
> > TrailingDistanceRange <- seq(0.01,0.02,by=0.01)
> >
> >
> > positionSizeLong = round(100000 / as.numeric(symbol.data$CLOSE[1]),-2)
> > positionSizeShort = - round(100000 / as.numeric(symbol.data$CLOSE[1]),-2)
> > txn.model <- 0
> > sltsltp.txn.fee <- 0
> >
> >
> > add.indicator(strategy.st,
> > name = "MACD",
> > arguments = list(x=Cl(eval(parse(text = "EURUSD")))),
> > label='macd')
> >
> > add.signal(strategy.st,name="sigCrossover",
> > arguments =
> > list(columns=c("macd.macd","signal.macd"),relationship="gt"),
> > label="macd.gt.signal")
> >
> > add.signal(strategy.st,name="sigCrossover",
> > arguments =
> > list(columns=c("macd.macd","signal.macd"),relationship="lt"),
> > label="macd.lt.signal")
> >
> > add.rule(strategy.st,
> > name='ruleSignal',
> > arguments = list(sigcol="macd.gt.signal",
> > sigval=TRUE,
> > prefer="Open",
> > orderqty= positionSizeLong,
> > ordertype='market',
> > orderside='long',
> > orderset='ocolong',
> > TxnFees = txn.model),
> > type='enter',
> > label='longenter',
> > enabled=TRUE
> > )
> >
> > add.rule(strategy.st,
> > name='ruleSignal',
> > arguments = list(sigcol="macd.lt.signal",
> > sigval=TRUE,
> > prefer="Open",
> > orderqty='all',
> > ordertype='market',
> > orderside='long',
> > orderset='ocolong',
> > TxnFees = txn.model),
> > type='exit',
> > label='longexit',
> > enabled=TRUE
> > )
> >
> > add.rule(strategy.st,name='ruleSignal',
> > arguments = list( sigcol="macd.lt.signal", sigval=TRUE,
> > replace=FALSE,
> > orderside='long',
> > ordertype='stoplimit',
> > tmult=TRUE,
> > threshold=quote( longStopLossDistance ),
> > orderqty='all',
> > orderset='ocolong',
> > TxnFees = txn.model),
> > type='chain', parent="longenter",
> > label='StopLossLong',
> > enabled=TRUE)
> >
> > add.rule(strategy.st, name = 'ruleSignal',
> > arguments=list(sigcol="macd.lt.signal" , sigval=TRUE,
> > replace=FALSE,
> > orderside='long',
> > ordertype='stoptrailing',
> > tmult=TRUE,
> > threshold=quote(longTrailingStopDistance),
> > orderqty='all',
> > orderset='ocolong',
> > TxnFees = txn.model),
> > type='chain', parent="longenter",
> > label='StopTrailingLong',
> > enabled=TRUE
> > )
> >
> > add.rule(strategy.st, name = "ruleSignal",
> > arguments = list(sigcol="macd.lt.signal",
> > sigval=TRUE,
> > ordertype="limit",
> > orderside="long",
> > replace=FALSE,
> > tmult=TRUE,
> > threshold=quote(longTakeProfitDistance),
> > orderqty="all",
> > orderset="ocolong",
> > TxnFees = txn.model),
> > type = "chain", parent="longenter",
> > label = "takeProfitLong",
> > enabled = FALSE
> > )
> >
> > add.rule(strategy.st,
> > name='ruleSignal',
> > arguments = list(sigcol="macd.lt.signal",
> > sigval=TRUE,
> > prefer="Open",
> > orderqty=positionSizeShort,
> > ordertype='market',
> > orderside='short',
> > orderset='ocoshort',
> > TxnFees = txn.model),
> > type='enter',
> > label='shortenter',
> > enabled=TRUE
> > )
> >
> > add.rule(strategy.st,
> > name='ruleSignal',
> > arguments = list(sigcol="macd.gt.signal",
> > sigval=TRUE,
> > prefer="Open",
> > orderqty='all',
> > ordertype='market',
> > orderside='short',
> > orderset='ocoshort',
> > TxnFees = txn.model),
> > type='exit',
> > label='shortexit',
> > enabled=TRUE
> > )
> >
> > add.rule(strategy.st,name='ruleSignal',
> > arguments = list( sigcol="macd.gt.signal", sigval=TRUE,
> > replace=FALSE,
> > orderside='short',
> > ordertype='stoplimit',
> > tmult=TRUE,
> > threshold=quote( shortStopLossDistance ),
> > orderqty='all',
> > orderset='ocoshort',
> > TxnFees = txn.model),
> > type='chain', parent="shortenter",
> > label='StopLossShort',
> > enabled=TRUE)
> >
> > add.rule(strategy.st, name = 'ruleSignal',
> > arguments=list(sigcol="macd.gt.signal" , sigval=TRUE,
> > replace=FALSE,
> > orderside='short',
> > ordertype='stoptrailing',
> > tmult=TRUE,
> > threshold=quote( shortTrailingStopDistance),
> > orderqty='all',
> > orderset='ocoshort',
> > TxnFees = txn.model),
> > type='chain', parent="shortenter",
> > label='StopTrailingShort',
> > enabled=TRUE
> > )
> >
> > add.rule(strategy.st, name = "ruleSignal",
> > arguments = list(sigcol="macd.gt.signal",
> > sigval=TRUE,
> > ordertype="limit",
> > orderside="short",
> > replace=FALSE,
> > tmult=TRUE,
> > threshold=quote( -shortTakeProfitDistance),
> > orderqty="all",
> > orderset="ocoshort",
> > TxnFees = txn.model),
> > type = "chain", parent="shortenter",
> > label = "takeProfitShort",
> > enabled = FALSE
> > )
> >
> >
> > add.distribution(strategy.st,
> > paramset.label = "MACD_OPT",
> > component.type = 'indicator',
> > component.label = "macd",
> > variable = list( nFast = macdFastMARange ),
> > label = "macdFastMARANGE")
> >
> >
> > add.distribution(strategy.st,
> > paramset.label = "MACD_OPT",
> > component.type = 'indicator',
> > component.label = "macd",
> > variable = list( nSlow = macdSlowMARange ),
> > label = "macdSlowMARANGE")
> >
> > add.distribution(strategy.st,
> > paramset.label = "MACD_OPT",
> > component.type = 'indicator',
> > component.label = "macd",
> > variable = list( nSig = macdSignalRange ),
> > label = "macdSignalRANGE")
> >
> > add.distribution.constraint(strategy.st,
> > paramset.label = 'MACD_OPT',
> > distribution.label.1 = 'macdFastMARANGE',
> > distribution.label.2 = 'macdSlowMARANGE',
> > operator = '<',
> > label = 'FastMA<SlowMA')
> >
> > add.distribution(strategy.st,
> > paramset.label = "MACD_OPT",
> > component.type = "chain",
> > component.label = "StopLossLong",
> > variable = list( threshold = StopLossDistanceRange ),
> > label = "StopLossLONG")
> >
> > add.distribution(strategy.st,
> > paramset.label = "MACD_OPT",
> > component.type = "chain",
> > component.label = "StopTrailingLong",
> > variable = list( threshold = TrailingDistanceRange ),
> > label = "StopTrailingLONG")
> >
> > add.distribution(strategy.st,
> > paramset.label = "MACD_OPT",
> > component.type = "chain",
> > component.label = "StopLossShort",
> > variable = list( threshold = StopLossDistanceRange ),
> > label = "StopLossSHORT")
> >
> > add.distribution(strategy.st,
> > paramset.label = "MACD_OPT",
> > component.type = "chain",
> > component.label = "StopTrailingShort",
> > variable = list( threshold = TrailingDistanceRange ),
> > label = "StopTrailingSHORT")
> >
> > add.distribution.constraint(strategy.st,
> > paramset.label = "MACD_OPT",
> > distribution.label.1 = "StopLossLONG",
> > distribution.label.2 = "StopLossSHORT",
> > operator = "==",
> > label = "StoplossEquality")
> >
> > add.distribution.constraint(strategy.st,
> > paramset.label = "MACD_OPT",
> > distribution.label.1 = "StopTrailingLONG",
> > distribution.label.2 = "StopTrailingSHORT",
> > operator = "==",
> > label = "TrailingStopEquality")
> >
> > summary(getStrategy(strategy.st))
> >
> >
> >
> > start.t <- Sys.time()
> > paramsetenv<-new.env()
> > cl <- snow::makeCluster(no.cores, type = "SOCK")
> > registerDoSNOW(cl)
> > results <- apply.paramset(strategy.st,paramset.label="MACD_OPT",
> > portfolio=strategy.st, account=strategy.st,nsamples=0,verbose
> > = FALSE,
> > audit=paramsetenv)
> > snow::stopCluster(cl)
> > finish.t <- Sys.time()
> > print(finish.t-start.t)
> >
> >
> >
> >
> >
> > If there are any problems with the code when reproducing, sorry. I ran it
> > on my computer and it did seem to work as it did normally.
> >
> >
> >
> > 2016-04-06 21:05 GMT+03:00 Brian G. Peterson <brian at braverock.com>:
> >
> >> On Wed, 2016-04-06 at 20:58 +0300, Jersey Fanatic wrote:
> >> > Hi everyone,
> >> >
> >> > I am trying to backtest a simple strategy of mine, but it took approx 10
> >> > hours of processing so I was wondering if it is normal.
> >> >
> >> > My computer has an i7 core 2.10 GHz, 8GB RAM with Windows 7 Ultimate
> >> OS. My
> >> > code uses doSNOW package for parallel processing and all the 8 cores, so
> >> > the CPU is %100 all the time for the complete 10 hours.
> >> >
> >> > The data the backtest is done on is M5 OHLC FX (24 hour) data for
> >> approx 1
> >> > year which makes a total of 58000 data points (counting OHLC as 1). The
> >> > strategy has 4 entry&exit and 4 stoploss&trailingstoploss&takeprofit
> >> rules.
> >> > 144 combinations of parameters are tested.
> >> >
> >> > I would just like to know if this time is normal for just 1 home-use
> >> > computer. And I would be grateful for any recommendations to speed up
> >> this
> >> > process to decrease computing time.
> >>
> >> No, this sounds very high.
> >>
> >> You haven't told us enough about what you're trying to do for anyone
> >> here to help you in a specific way. You'd need to provide a minimal
> >> reproducible example.
> >>
> >> However, we typically use a benchmark time of one core minute per day
> >> per instrument on tick data (millions of observations per day).
> >>
> >> Is seems likely that you are doing too much path dependent looping,
> >> copying, or both.
> >>
> >> Regards,
> >>
> >> Brian
> >>
> >>
> >
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-SIG-Finance at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>
More information about the R-SIG-Finance
mailing list