[R-SIG-Finance] quantstrat: getting an error when using many symbols and %-based order sizing function

Sergey Pisarenko drseergio at gmail.com
Mon Feb 27 20:28:27 CET 2012 

Garrett,

Thanks for your insight on this issue. This was totally the problem
here. The different symbols had data for different date spans. For
some symbols there were NAs during certain spans (e.g. stock did not
exist yet). The right solution to me seems to either use data that
overlaps (alignSymbols from qmao does that) or skip transactions for
those symbols that have a gap on the particular dates.

/Sergey

On Mon, Feb 27, 2012 at 9:55 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
> Garett,
>
> Thanks, great point. Total blunder on my side. I will look into this further.
>
> /Sergey
>
> On Mon, Feb 27, 2012 at 9:45 AM, G See <gsee000 at gmail.com> wrote:
>> Sergey,
>>
>> Do you have data for 2009 for AAT, because yahoo doesn't.
>>
>>> start(AAT)
>> [1] "2011-01-13"
>>
>> HTH,
>> Garrett
>>
>>
>> On Mon, Feb 27, 2012 at 12:18 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>> Hi,
>>>
>>> Going even deeper I see that the problem occurs inside of .updatePosPL
>>> function (blotter) where dates are extracted:
>>>
>>>> updatePortf(Portfolio='p', Dates=paste('2009-01-01', '::', '2009-04-06', sep=''))
>>> Error in if (nzchar(intervals[1])) s <- as.POSIXlt(do.call(firstof,
>>> parse.side(intervals[1]))) :
>>>  argument is of length zero
>>>> traceback()
>>> 4: xts:::.parseISO8601(Dates)
>>> 3: first(xts:::.parseISO8601(Dates))
>>> 2: .updatePosPL(Portfolio = pname, Symbol = as.character(symbol),
>>>       Dates = Dates, Prices = Prices, ... = ...)
>>> 1: updatePortf(Portfolio = "p", Dates = paste("2009-01-01", "::",
>>>       "2009-04-06", sep = ""))
>>>
>>> I have looked in source code  blotter / R / updatePortf.R and I see
>>> the following lines:
>>>        startDate = first(xts:::.parseISO8601(Dates))$first.time-1 #does this
>>> need to be a smaller/larger delta for millisecond data?
>>>        endDate   = last(xts:::.parseISO8601(Dates))$last.time
>>>
>>> Apparently, the problem occurs in the "startDate" calculation. I tried
>>> manually calling xts:::.parseISO8601('2009-04-06') and
>>> xts:::.parseISO8601('::2009-04-06') and it works OK.
>>>
>>> Any ideas why xts:::.parse would be failing during script execution?
>>>
>>> --
>>> Kind Regards,
>>> Sergey Pisarenko.
>>>
>>> On Mon, Feb 27, 2012 at 6:51 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>> Hi,
>>>>
>>>> The specific line this occurs at is:
>>>>
>>>>    dummy <- updatePortf(Portfolio=portfolio, Dates=paste('::',
>>>> as.Date(timestamp), sep=''))
>>>>
>>>> The function is inside of the sizing function. I have checked that
>>>> timestamp is valid when this function is called so the problem has to
>>>> do with some sort of a state problem in the portfolio.
>>>>
>>>> /Sergey
>>>>
>>>> On Mon, Feb 27, 2012 at 6:41 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> Sorry for not following up sooner as I had been away for a week. I
>>>>> have been able to further narrow down the problem. It occurs in the
>>>>> sizing function and only when there are too many symbols. I have
>>>>> further trimmed down the sample code.
>>>>>
>>>>> I will continue troubleshooting this function. Do you have any ideas
>>>>> that would point me in the right direction?
>>>>>
>>>>> /Sergey
>>>>>
>>>>> On Fri, Feb 17, 2012 at 9:11 PM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>>>> Garett,
>>>>>>
>>>>>> Thank you for the response and looking into this. The same is observed
>>>>>> when I use MySQL (I keep a copy of data on a local DB). Yahoo does not
>>>>>> have data for all symbols as well so this might explain some of the
>>>>>> 404s.
>>>>>>
>>>>>> /Sergey
>>>>>>
>>>>>> On Fri, Feb 17, 2012 at 12:08 PM, G See <gsee000 at gmail.com> wrote:
>>>>>>> I've been having trouble with yahoo over the last 24 hours or so.
>>>>>>> Sometimes, it returns data; sometimes it doesn't
>>>>>>>
>>>>>>>> watched <- c("A", "AA", "AAN", "AAP", "AAT", "AAV")
>>>>>>>> getSymbols(watched, src='yahoo', verbose=FALSE)
>>>>>>> pausing 1 second between requests for more than 5 symbols
>>>>>>> pausing 1 second between requests for more than 5 symbols
>>>>>>> [1] "A"   "AA"  "AAN" "AAP" "AAT" "AAV"
>>>>>>>> getSymbols(watched, src='yahoo', verbose=FALSE)
>>>>>>> Error in download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m,  :
>>>>>>>  cannot open URL
>>>>>>> 'http://chart.yahoo.com/table.csv?s=AAT&a=0&b=01&c=2007&d=1&e=17&f=2012&g=d&q=q&y=0&z=AAT&x=.csv'
>>>>>>> In addition: Warning message:
>>>>>>> In download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m,  :
>>>>>>>  cannot open: HTTP status was '404 Not Found'
>>>>>>>
>>>>>>>
>>>>>>> HTH,
>>>>>>> Garrett
>>>>>>>
>>>>>>> On Fri, Feb 17, 2012 at 2:01 PM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>>>>>> Greetings to the R finance community,
>>>>>>>>
>>>>>>>> I have written a sizing function (pretty much a slight modification of
>>>>>>>> a function from Guy Yollin's slides). The problem is that it fails
>>>>>>>> whenever I use many symbols. It works fine for one or few symbols.
>>>>>>>>
>>>>>>>> The problem manifests itself inside of the sizing function, when
>>>>>>>> updatePortf is run. The error I get is:
>>>>>>>>> source('tainted_2.R')
>>>>>>>> Error in if (nzchar(intervals[1])) s <- as.POSIXlt(do.call(firstof,
>>>>>>>> parse.side(intervals[1]))) :
>>>>>>>>  argument is of length zero
>>>>>>>>
>>>>>>>> Traceback:
>>>>>>>>> traceback()
>>>>>>>> 13: xts:::.parseISO8601(Dates)
>>>>>>>> 12: first(xts:::.parseISO8601(Dates))
>>>>>>>> 11: .updatePosPL(Portfolio = pname, Symbol = as.character(symbol),
>>>>>>>>        Dates = Dates, Prices = Prices, ... = ...)
>>>>>>>> 10: updatePortf(Portfolio = portfolio, Dates = paste("::", as.Date(timestamp),
>>>>>>>>        sep = "")) at tainted_2.R#100
>>>>>>>> 9: osFUN(strategy = strategy, data = data, timestamp = timestamp,
>>>>>>>>       orderqty = orderqty, ordertype = ordertype, orderside = orderside,
>>>>>>>>       portfolio = portfolio, symbol = symbol, ... = ..., ruletype = ruletype,
>>>>>>>>       orderprice = as.numeric(orderprice))
>>>>>>>> 8: function (data = mktdata, timestamp, sigcol, sigval, orderqty = 0,
>>>>>>>>       ordertype, orderside = NULL, threshold = NULL, tmult = FALSE,
>>>>>>>>       replace = TRUE, delay = 1e-04, osFUN = "osNoOp", pricemethod =
>>>>>>>> c("market",
>>>>>>>>           "opside", "active"), portfolio, symbol, ..., ruletype,
>>>>>>>>       TxnFees = 0, prefer = NULL, sethold = FALSE)
>>>>>>>> ...
>>>>>>>> 7: do.call(fun, .formals)
>>>>>>>> 6: ruleProc(strategy$rules[[type]], timestamp = timestamp, path.dep = path.dep,
>>>>>>>>       mktdata = mktdata, portfolio = portfolio, symbol = symbol,
>>>>>>>>       ruletype = type, mktinstr = mktinstr, ...)
>>>>>>>> 5: applyRules(portfolio = portfolio, symbol = symbol, strategy = strategy,
>>>>>>>>       mktdata = mktdata, Dates = NULL, indicators = sret$indicators,
>>>>>>>>       signals = sret$signals, parameters = parameters, ..., path.dep = TRUE)
>>>>>>>> 4: applyStrategy(strategy = "s", portfolios = "p")
>>>>>>>> 3: eval.with.vis(expr, envir, enclos)
>>>>>>>> 2: eval.with.vis(ei, envir)
>>>>>>>> 1: source("tainted_2.R")
>>>>>>>>
>>>>>>>> The sizing function looks like:
>>>>>>>>
>>>>>>>> osPercentEquity <- function(timestamp, orderqty, portfolio, symbol,
>>>>>>>> ruletype, ...) {
>>>>>>>>  posn <- getPosQty(portfolio, symbol, timestamp)
>>>>>>>>  if (posn == 0) {
>>>>>>>>    tempPortfolio <- getPortfolio(portfolio)
>>>>>>>>    dummy <- updatePortf(Portfolio=portfolio, Dates=paste('::',
>>>>>>>> as.Date(timestamp), sep=''))     # <<<<<< this is where it fails
>>>>>>>>    trading.pl <- sum(getPortfolio(portfolio)$summary$Net.Trading.PL)
>>>>>>>>    assign(paste("portfolio.", portfolio, sep=""), tempPortfolio, pos=.blotter)
>>>>>>>>    total.equity <- equity + trading.pl
>>>>>>>>    tradeSize <- total.equity * prcTrade
>>>>>>>>    ClosePrice <- as.numeric(Cl(mktdata[timestamp,]))
>>>>>>>>    orderqty <- sign(orderqty) * round(tradeSize / ClosePrice)
>>>>>>>>    return(orderqty)
>>>>>>>>  }
>>>>>>>> }
>>>>>>>>
>>>>>>>> The add.rule looks like:
>>>>>>>> s <- add.rule(strategy=s, name='ruleSignal',
>>>>>>>> arguments=list(data=quote(mktdata), sigcol='buySig', sigval=TRUE,
>>>>>>>> orderqty=qtyDef, ordertype='market', orderside=NULL, threshold=NULL,
>>>>>>>> osFUN='osPercentEquity', ruletype='enter'), type='enter')
>>>>>>>>
>>>>>>>> Traceback suggests that there is a problem with the date format. I
>>>>>>>> have inserted print statements and verified that the date itself is
>>>>>>>> fine. It is also weird that the issue does not occur when using few
>>>>>>>> symbols.
>>>>>>>>
>>>>>>>> I have attached runnable code that reproduces the problem. Please
>>>>>>>> excuse any apparent inefficiencies you see. I would like to get
>>>>>>>> strategy working first and then optimize for performance.
>>>>>>>>
>>>>>>>> Is there something wrong with the approach I have taken? Once the
>>>>>>>> number of symbols is reduced the problem does not trigger.
>>>>>>>>
>>>>>>>> Truly appreciate your help.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Kind Regards,
>>>>>>>> Sergey Pisarenko.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> R-SIG-Finance at r-project.org mailing list
>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>>>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>>>>>> -- Also note that this is not the r-help list where general R questions should go.

More information about the R-SIG-Finance mailing list