[R-SIG-Finance] quantstrat help - simple combine error using windows and walk.forward
Derek Wong
treydog999 at gmail.com
Wed Oct 1 04:55:45 CEST 2014
Hi Josh,
Sorry about the HTML mistake, I am new to using something like
r-sig-finance mailing list, I didn't realize I was posting
incorrectly. In regards to the personal correspondence, I didnt full
understand that it was operating system independent, I thought that
because windows has problems with MC solutions that could have been
the issue. I have tried doing using the doRedis package and it just
runs for hours on end., but never showing a result. I am guessing it
should not take 4+ hours to run this, but i can not see and additional
output in R so i am not sure how far along it got.. Although the
server console does show
10 changes in 300 seconds. Saving...
fork operation complete
Background saving terminated success.
I have also tried using registerdoSEQ() think that was a guaranteed
way of seeing the results however i recieve an error. here is the
output. I have checked if i had the iterators package installed and
loaded which I do. So i do not know where it is trying to call this
function. I did a search for help on iter assuming it was a built in
function but did not find anything.
> registerDoSEQ()
>
> result <- walk.forward(pairStrat,
+ paramset.label = "BBOPT",
+ portfolio.st = portfolio1.st,
+ account.st = account.st,
+ period = "months",
+ k.training = 6,
+ k.testing = 3,
+ obj.func = my.obj.func,
+ obj.args = list(x=quote(result$apply.paramset)),
+ audit.prefix = "wfa",
+ anchored = FALSE,
+ verbose = TRUE)
[1] "=== training BBOPT on 2009-01-02/2009-06-30"
Error in eval(expr, envir, enclos) : could not find function "iter"
All Code with appended doRedis and doSEQ -------- (hopefully this
posts correctly)
#Kindly contributed to quantstrat by Garrett See
#code borrowed heavily from existing quantstrat demos
# This is a simple pairs trading example intended to illustrate how you can
# extend existing quantstrat functionality. It uses addPosLimits to specify
# levels and position limits, and shows how to pass a custom order sizing
# function to osFUN
# Note that it would be easier to build a spread first and treat it as a single
# instrument instead of dealing with a portfolio of stocks.
## given 2 stocks, calculate the ratio of their notional values. If the ratio
# falls below it's 2 stdev band, then when it crosses back above it, buy stock 1
# and sell stock 2. If the ratio rises above it's 2 stdev band, then when it
# crosses back below it, sell stock 1 and buy stock 2. If the ratio crosses
# its moving average, then flatten any open positions.
# The Qty of Stock A that it buys (sells) = MaxPos / lvls
# The Qty of Stock B that is sells (buys) = MaxPos * Ratio / lvls
require(quantstrat)
suppressWarnings(rm("order_book.pair1",pos=.strategy))
suppressWarnings(rm("account.pairs", "portfolio.pair1", pos=.blotter))
suppressWarnings(rm("initDate", "endDate", "startDate", "initEq", "SD", "N",
"symb1", "symb2", "portfolio1.st", "account.st",
"pairStrat", "out1"))
##### PLACE DEMO AND TEST DATES HERE #################
#
#if(isTRUE(options('in_test')$in_test))
# # use test dates
# {initDate="2011-01-01"
# endDate="2012-12-31"
# } else
# # use demo defaults
# {initDate="1999-12-31"
# endDate=Sys.Date()}
initDate <- '2009-01-01'
endDate <- '2011-05-01'
startDate <- '2009-01-02'
initEq <- 100000
SD <- 2
N <- 20
MaxPos <- 1500 #max position in stockA;
# max position in stock B will be max * ratio, i.e. no hard position limit in
# Stock B
lvls <- 3 #how many times to fade; Each order's qty will = MaxPos/lvls
symb1 <- 'SPY' #change these to try other pairs
symb2 <- 'DIA' #if you change them, make sure position limits still make sense
portfolio1.st <- 'pair1'
account.st <- 'pairs'
getSymbols(c(symb1, symb2), from=startDate, to=endDate, adjust=TRUE)
# The following function is used to make sure the timestamps of all symbols are
# the same deletes rows where one of the stocks is missing data
alignSymbols <- function(symbols, env=.GlobalEnv) {
# This is a simplified version of qmao::alignSymbols()
if (length(symbols) < 2)
stop("Must provide at least 2 symbols")
if (any(!is.character(symbols)))
stop("Symbols must be vector of character strings.")
ff <- get(symbols[1],env=env)
for (sym in symbols[-1]) {
tmp.sym <- get(sym,env=env)
ff <- merge(ff, tmp.sym, all=FALSE)
}
for (sym in symbols) {
assign(sym,ff[,grep(sym, colnames(ff))], env=env)
}
symbols
}
alignSymbols(c(symb1, symb2))
# Define Instruments
currency("USD")
stock(symb1, currency="USD", multiplier=1)
stock(symb2, currency="USD", multiplier=1)
# Initialize Portfolio, Account, and Orders
initPortf(name=portfolio1.st, c(symb1,symb2), initDate=initDate)
initAcct(account.st, portfolios=portfolio1.st, initDate=initDate, initEq=initEq)
initOrders(portfolio=portfolio1.st, initDate=initDate)
# osFUN will need to know which symbol is leg 1 and which is leg 2 as well as
# what the values are for MaxPos and lvls. So, create a slot in portfolio to
# hold this info.
pair <- c(1, 2, MaxPos, lvls)
names(pair) <- c(symb1, symb2, "MaxPos", "lvls")
.blotter[[paste('portfolio', portfolio1.st, sep='.')]]$pair <- pair
# Create initial position limits and levels by symbol
# allow 3 entries for long and short if lvls=3.
addPosLimit(portfolio=portfolio1.st, timestamp=initDate, symbol=symb1,
maxpos=MaxPos, longlevels=lvls, minpos=-MaxPos, shortlevels=lvls)
addPosLimit(portfolio=portfolio1.st, timestamp=initDate, symbol=symb2,
maxpos=MaxPos, longlevels=lvls, minpos=-MaxPos, shortlevels=lvls)
# Create a strategy object
pairStrat <- strategy('pairStrat')
# Indicator function
calcRatio <- function(x) {
#returns the ratio of notional close prices for 2 symbols
x1 <- get(x[1])
x2 <- get(x[2])
mult1 <- getInstrument(x[1])$multiplier
mult2 <- getInstrument(x[2])$multiplier
rat <- (mult1 * Cl(x1)) / (mult2 * Cl(x2))
colnames(rat) <- 'Ratio'
rat
}
# Indicator used for determining entry/exits
Ratio <- calcRatio(c(symb1[1], symb2[1]))
# Store hedge ratio in portfolio so that it's available for order sizing
# function. In this example, the hedge ratio happens to be the same as the
# Ratio indicator.
.blotter[[paste('portfolio',portfolio1.st,sep='.')]]$HedgeRatio <- Ratio
#and make a function to get the most recent HedgeRatio
getHedgeRatio <- function(portfolio, timestamp) {
portf <- getPortfolio(portfolio)
timestamp <- format(timestamp,"%Y-%m-%d %H:%M:%S")
# above line ensures you don't get last value of next day if using intraday
# data and timestamp=midnight
toDate <- paste("::", timestamp, sep="")
Ratio <- last(portf$HedgeRatio[toDate])
as.numeric(Ratio)
}
# Create an indicator - BBands on the Ratio
pairStrat <- add.indicator(strategy=pairStrat, name = "calcRatio",
arguments=list(x=c(symb1,symb2)))
pairStrat <- add.indicator(strategy=pairStrat, name = "BBands",
arguments=list(HLC=quote(Ratio), sd=SD, n=N,
maType='SMA'),
label = "BBands")
#applyIndicators(strategy=pairStrat,mktdata=get(symb1[1])) #for debugging
# Create signals - buy when crossing lower band from below, sell when crossing
# upper band from above, flatten when crossing mavg from above or from below
pairStrat <- add.signal(strategy=pairStrat, name="sigCrossover",
arguments=list(columns=c("Ratio","up"),
relationship="lt"),
label="cross.up")
pairStrat <- add.signal(strategy=pairStrat, name="sigCrossover",
arguments=list(columns=c("Ratio","dn"),
relationship="gt"),
label="cross.dn")
pairStrat <- add.signal(strategy=pairStrat, name="sigCrossover",
arguments=list(columns=c("Ratio","mavg"),
relationship="lt"),
label="cross.mid.fa")
pairStrat <- add.signal(strategy=pairStrat, name="sigCrossover",
arguments=list(columns=c("Ratio","mavg"),
relationship="gt"),
label="cross.mid.fb")
# make an order sizing function
#######################_ORDER SIZING FUNCTION_##################################
# check to see which stock it is. If it's the second stock, reverse orderqty and
# orderside
osSpreadMaxPos <- function (data, timestamp, orderqty, ordertype, orderside,
portfolio, symbol, ruletype, ..., orderprice) {
portf <- getPortfolio(portfolio)
#check to make sure pair slot has the things needed for this function
if (!any(portf$pair == 1) && !(any(portf$pair == 2)))
stop('pair must contain both values 1 and 2')
if (!any(names(portf$pair) == "MaxPos") || !any(names(portf$pair) == "lvls"))
stop('pair must contain MaxPos and lvls')
if (portf$pair[symbol] == 1) legside <- "long"
if (portf$pair[symbol] == 2) legside <- "short"
MaxPos <- portf$pair["MaxPos"]
lvls <- portf$pair["lvls"]
ratio <- getHedgeRatio(portfolio, timestamp)
pos <- getPosQty(portfolio, symbol, timestamp)
PosLimit <- getPosLimit(portfolio, symbol, timestamp)
qty <- orderqty
if (legside == "short") {#symbol is 2nd leg
## Comment out next line to use equal ordersizes for each stock.
addPosLimit(portfolio=portfolio, timestamp=timestamp, symbol=symbol,
maxpos=round(MaxPos*ratio,0), longlevels=lvls,
minpos=round(-MaxPos*ratio,0), shortlevels=lvls)
##
qty <- -orderqty #switch orderqty for Stock B
}
if (qty > 0) orderside = 'long'
if (qty < 0) orderside = 'short'
orderqty <- osMaxPos(data=data,timestamp=timestamp, orderqty=qty,
ordertype=ordertype, orderside=orderside,
portfolio=portfolio, symbol=symbol, ruletype=ruletype,
...)
#Add the order here instead of in the ruleSignal function
if (!is.null(orderqty) & !orderqty == 0 & !is.null(orderprice)) {
addOrder(portfolio=portfolio, symbol=symbol,
timestamp=timestamp, qty=orderqty, price=as.numeric(orderprice),
ordertype=ordertype, side=orderside, replace=FALSE,
status="open", ...=...)
}
return(0) #so that ruleSignal function doesn't also try to place an order
}
################################################################################
# Create entry and exit rules for longs and for shorts. Both symbols will get
# the same buy/sell signals, but osMaxPos will reverse those for the second
# symbol.
# orderqty's are bigger than PosLimits allow. osMaxPos will adjust the orderqty
# down to 1/3 the max allowed. (1/3 is because we are using 3 levels in
# PosLimit)
pairStrat <- add.rule(strategy=pairStrat, name='ruleSignal',
arguments=list(sigcol="cross.dn", sigval=TRUE,
orderqty=1e6, ordertype='market',
orderside=NULL, osFUN='osSpreadMaxPos'),
type='enter')
pairStrat <- add.rule(strategy=pairStrat, name='ruleSignal',
arguments=list(sigcol="cross.up", sigval=TRUE,
orderqty=-1e6, ordertype='market',
orderside=NULL, osFUN='osSpreadMaxPos'),
type='enter')
pairStrat <- add.rule(strategy=pairStrat, name='ruleSignal',
arguments=list(sigcol="cross.mid.fb", sigval=TRUE,
orderqty='all', ordertype='market',
orderside=NULL),
type='exit')
pairStrat <- add.rule(strategy=pairStrat, name='ruleSignal',
arguments=list(sigcol="cross.mid.fa", sigval=TRUE,
orderqty='all', ordertype='market',
orderside=NULL),
type='exit')
## for debugging
# applySignals(strategy=pairStrat,
# mktdata=applyIndicators(strategy=pairStrat, mktdata=get(symb1)))
##
out1<-applyStrategy(strategy=pairStrat, portfolios=portfolio1.st)
updatePortf(Portfolio=portfolio1.st,
Dates=paste("::", as.Date(Sys.time()), sep=''))
updateAcct(account.st, Dates=paste(startDate, endDate, sep="::"))
updateEndEq(account.st, Dates=paste(startDate, endDate, sep="::"))
getEndEq(account.st, Sys.time())
dev.new()
chart.Posn(Portfolio=portfolio1.st, Symbol=symb1)
dev.new()
chart.Posn(Portfolio=portfolio1.st, Symbol=symb2)
dev.new()
chartSeries(Cl(get(symb1))/Cl(get(symb2)), TA="addBBands(n=N,sd=SD)")
ret1 <- PortfReturns(account.st)
ret1$total <- rowSums(ret1)
#ret1
if("package:PerformanceAnalytics" %in% search() ||
require("PerformanceAnalytics",quietly=TRUE)) {
# getSymbols("SPY", from='1999-01-01')
# SPY.ret <- Return.calculate(SPY$SPY.Close)
# tmp <- merge(SPY.ret,ret1$total,all=FALSE)
dev.new()
charts.PerformanceSummary(ret1$total, geometric=FALSE, wealth.index=TRUE)
}
###############################################################################
# R (http://r-project.org/) Quantitative Strategy Model Framework
#
# Package Copyright (c) 2009-2012
# Peter Carl, Dirk Eddelbuettel, Brian G. Peterson, Jeffrey Ryan, and
Joshua Ulrich
#
# This library is distributed under the terms of the GNU Public License (GPL)
# for full details see the file COPYING
#
# $Id$
#
###############################################################################
##### PLACE THIS BLOCK AT END OF DEMO SCRIPT ###################
# book = getOrderBook(port)
# stats = tradeStats(port)
# rets = PortfReturns(acct)
################################################################
#Distribution Initialization
pairStrat <- add.distribution(strategy = pairStrat,
paramset.label ="BBOPT",
component.type = "indicator",
component.label = "BBands",
variable = list(n = seq(10,200, by = 10)),
label = "n")
pairStrat <- add.distribution(strategy = pairStrat,
paramset.label ="BBOPT",
component.type = "indicator",
component.label = "BBands",
variable = list(sd = seq(1,5,by = 0.5)),
label = "sd")
# Objective Function
my.obj.func <- function(x)
{
# pick one of the following objective functions (uncomment)
#return(max(x$tradeStats$Max.Drawdown) == x$tradeStats$Max.Drawdown)
return(max(x$tradeStats$Net.Trading.PL) == x$tradeStats$Net.Trading.PL)
}
#WFA - Comment out selected method
# require(doParallel)
# cl <- makePSOCKcluster(2)
# registerDoParallel(cl)
#
# require(doRedis)
# registerDoRedis('jobs')
# startLocalWorkers(n=2, queue='jobs')
# print(getDoParWorkers())
registerDoSEQ()
result <- walk.forward(pairStrat,
paramset.label = "BBOPT",
portfolio.st = portfolio1.st,
account.st = account.st,
period = "months",
k.training = 6,
k.testing = 3,
obj.func = my.obj.func,
obj.args = list(x=quote(result$apply.paramset)),
audit.prefix = "wfa",
anchored = FALSE,
verbose = TRUE)
I
On Tue, Sep 30, 2014 at 10:34 AM, Joshua Ulrich <josh.m.ulrich at gmail.com> wrote:
> Tue, Sep 30, 2014 at 10:25 AM, Derek Wong <treydog999 at gmail.com> wrote:
>> Mark,
>>
>> Thank you for the reply. I really appreciate you taking the time to look at
>> it. However in my previous email/post and my current code I do not see a
>> line that shows
>>
>> portfolio1.st <- 'pair1'account.st <- 'pairs'
>>
> You're posting in HTML, which the posting guild tells you not to do.
> The list server must convert your HTML into plain text, which often
> mangles your original message. For example, look at how your message
> was received at Nabble:
> http://r.789695.n4.nabble.com/quantstrat-help-simple-combine-error-using-windows-and-walk-forward-td4697468.html
>
>> they should be two separate lines. Either way I have run the entire code
>> and will show the area I am focusing on. Which is the end using the
>> walk.forward() function, as everything else in the code runs perfectly for
>> me using R 3.1.1 in windows or ubuntu.
>>
> As I said in our personal correspondence, "the problem is that the
> calcRatio function is not being exported to the nodes. This is
> probably related to bug #5814
> (https://r-forge.r-project.org/tracker/index.php?func=detail&aid=5814&group_id=316&atid=1269)."
>
> This is going to be a problem with a socket cluster on any operating
> system. It will work if you don't run in parallel, or if you run on
> almost anything other than a socket cluster (e.g. redis, fork
> [multicore], etc).
>
> Best,
> Josh
>
>
>> Code I am focusing on:
>>
>> # Objective Function
>> my.obj.func <- function(x)
>> {
>> # pick one of the following objective functions (uncomment)
>>
>> #return(max(x$tradeStats$Max.Drawdown) == x$tradeStats$Max.Drawdown)
>>
>> return(max(x$tradeStats$Net.Trading.PL) == x$tradeStats$Net.Trading.PL)
>>
>> }
>>
>> #WFA
>> require(doParallel)
>> cl <- makePSOCKcluster(2)
>> registerDoParallel(cl)
>>
>> result <- walk.forward(pairStrat,
>> paramset.label = "BBOPT",
>> portfolio.st = portfolio1.st,
>> account.st = account.st,
>> period = "months",
>> k.training = 6,
>> k.testing = 3,
>> obj.func = my.obj.func,
>> obj.args = list(x=quote(result$apply.paramset)),
>> audit.prefix = "wfa",
>> anchored = FALSE,
>> verbose = TRUE)
>>
>>
>> Output below:
>>> # Objective Function
>>> my.obj.func <- function(x)
>> + {
>> + # pick one of the following objective functions (uncomment)
>> +
>> + #return(max(x$tradeStats$Max.Drawdown) == x$tradeStats$Max.Drawdown)
>> +
>> + return(max(x$tradeStats$Net.Trading.PL) == x$tradeStats$Net.Trading.PL)
>> +
>> + }
>>>
>>> #WFA
>>> require(doParallel)
>>> cl <- makePSOCKcluster(2)
>>> registerDoParallel(cl)
>>>
>>> result <- walk.forward(pairStrat,
>> + paramset.label = "BBOPT",
>> + portfolio.st = portfolio1.st,
>> + account.st = account.st,
>> + period = "months",
>> + k.training = 6,
>> + k.testing = 3,
>> + obj.func = my.obj.func,
>> + obj.args = list(x=quote(result$apply.paramset)),
>> + audit.prefix = "wfa",
>> + anchored = FALSE,
>> + verbose = TRUE)
>> [1] "=== training BBOPT on 2009-01-02/2009-06-30"
>> automatically exporting the following variables from the local environment:
>> account, account.st, calc, env.instrument, mktdata, orderbook,
>> paramset.label, portfolio, portfolio.st, strategy, user.args, user.func
>> explicitly exporting variables(s): clone.portfolio, clone.orderbook,
>> install.param.combo
>> numValues: 180, numResults: 0, stopped: TRUE
>> got results for task 1
>> numValues: 180, numResults: 1, stopped: TRUE
>> returning status FALSE
>> got results for task 2
>> <SNIP>
>> got results for task 180
>> numValues: 180, numResults: 180, stopped: TRUE
>> first call to combine function
>> evaluating call object to combine results:
>> fun(result.1, result.2, result.3, result.4, result.5, result.6,
>> <SNIP>
>> result.177, result.178, result.179, result.180)
>> error calling combine function:
>> <simpleError in fun(result.1, result.2, result.3, result.4, result.5,
>> result.6, result.7, result.8, result.9, result.10, result.11,
>> result.12, result.13, result.14, result.15, result.16, result.17,
>> result.18, result.19, result.20, result.21, result.22,
>> <SNIP>
>> result.171, result.172, result.173, result.174, result.175, result.176,
>> result.177, result.178, result.179, result.180): attempt to select less
>> than one element>
>> Error in walk.forward(pairStrat, paramset.label = "BBOPT", portfolio.st =
>> portfolio1.st, :
>> obj.func() returned empty result
>> In addition: Warning messages:
>> 1: In e$fun(obj, substitute(ex), parent.frame(), e$data) :
>> already exporting variable(s): env.instrument
>> 2: In max(x$tradeStats$Net.Trading.PL) :
>> no non-missing arguments to max; returning -Inf
>>
>>
>>
>> Thanks again
>>
>> Derek
>>
>> On Tue, Sep 30, 2014 at 8:20 AM, Mark Knecht <markknecht at gmail.com> wrote:
>>
>>> On Tue, Sep 30, 2014 at 12:34 AM, Derek Wong <treydog999 at gmail.com> wrote:
>>> > Hello,
>>> >
>>> > I have been able to reproduce this error in Ubuntu based system as well
>>> not
>>> > just on windows. I was wondering if anyone had any insights or ways to to
>>> > solve this or create a work around. If there is any additional
>>> information
>>> > you require let me know. Please I would really appreciate help on this.
>>> It
>>> > is totally out of my depth and I am hoping that this fine community can
>>> > help.
>>> >
>>>
>>> Derek,
>>> I have no time to do any debugging of this but the first error on my
>>> Gentoo system running RStudio and R-3.1.1 is actually this line:
>>>
>>> portfolio1.st <- 'pair1'account.st <- 'pairs'
>>>
>>> which does look like bad code to me.
>>>
>>> One 'problem' with a number of demos that are out there is that the demo
>>> author may be using the development version of a specific package which
>>> then results in end user types (like me) having problems.
>>>
>>> I suggest you run this code line-by-line and then post back a few lines
>>> preceding the breakdown. Maybe someone can give better input than
>>> I can. Also, please edit/snip your responses to add focus to the exact
>>> issue.
>>>
>>> Good luck,
>>> Mark
>>>
>>> > On Wed, Sep 24, 2014 at 9:36 PM, Derek Wong <treydog999 at gmail.com>
>>> wrote:
>>> >
>>> >> Hello,
>>> >>
>>> >> I am having a lot of trouble with using walk.forward.in windows. i get
>>> >> the following error. This may be related to bug #5814. If someone could
>>> >> please help me i would really appreciate it. The reproducible code
>>> below is
>>> >> based on the pair_trade example. Thanks.
>>> >>
>>> >> error calling combine function:
>>> >> <simpleError in fun(result.1, result.2, result.3, result.4, result.5,
>>> >> result.6, result.7, result.8, result.9, result.10, result.11,
>>> <SNIP>
>>> >> result.172, result.173, result.174, result.175, result.176,
>>> result.177,
>>> >> result.178, result.179, result.180): attempt to select less than one
>>> >> element>
>>> >> Error in walk.forward(pairStrat, paramset.label = "BBOPT", portfolio.st
>>> =
>>>
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-SIG-Finance at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions should go.
>
>
> --
> Joshua Ulrich | about.me/joshuaulrich
> FOSS Trading | www.fosstrading.com
More information about the R-SIG-Finance
mailing list