[R-SIG-Finance] quantstrat: getting an error when using many symbols and %-based order sizing function
G See
gsee000 at gmail.com
Tue Feb 28 12:45:25 CET 2012
You could also just ignore stocks that don't have enough rows
(whatever that means to you). Maybe something like:
watched <- c("A", "AA", "AAN", "AAP", "AAT", "AAV")
nonIncompleteStocks <- function(Symbols, n=1000) {
# Symbols is vector of names of stocks
# n is an arbitrary number. Only stocks that have at least as many rows as
# the median minus n will be kept. A larger value will keep more stocks
require(qmao)
mypf <- makePriceFrame(getSymbols(Symbols, src='yahoo'),
na.omit=FALSE, notional=FALSE)
lengths <- sort(apply(mypf, 2, function(x) length(x[!is.na(x)])))
names(lengths)[lengths >= median(lengths) - n]
}
nonIncompleteStocks(watched)
#####################################################
-Garrett
On Mon, Feb 27, 2012 at 1:28 PM, Sergey Pisarenko <drseergio at gmail.com> wrote:
> Garrett,
>
> Thanks for your insight on this issue. This was totally the problem
> here. The different symbols had data for different date spans. For
> some symbols there were NAs during certain spans (e.g. stock did not
> exist yet). The right solution to me seems to either use data that
> overlaps (alignSymbols from qmao does that) or skip transactions for
> those symbols that have a gap on the particular dates.
>
> /Sergey
>
> On Mon, Feb 27, 2012 at 9:55 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>> Garett,
>>
>> Thanks, great point. Total blunder on my side. I will look into this further.
>>
>> /Sergey
>>
>> On Mon, Feb 27, 2012 at 9:45 AM, G See <gsee000 at gmail.com> wrote:
>>> Sergey,
>>>
>>> Do you have data for 2009 for AAT, because yahoo doesn't.
>>>
>>>> start(AAT)
>>> [1] "2011-01-13"
>>>
>>> HTH,
>>> Garrett
>>>
>>>
>>> On Mon, Feb 27, 2012 at 12:18 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>> Hi,
>>>>
>>>> Going even deeper I see that the problem occurs inside of .updatePosPL
>>>> function (blotter) where dates are extracted:
>>>>
>>>>> updatePortf(Portfolio='p', Dates=paste('2009-01-01', '::', '2009-04-06', sep=''))
>>>> Error in if (nzchar(intervals[1])) s <- as.POSIXlt(do.call(firstof,
>>>> parse.side(intervals[1]))) :
>>>> argument is of length zero
>>>>> traceback()
>>>> 4: xts:::.parseISO8601(Dates)
>>>> 3: first(xts:::.parseISO8601(Dates))
>>>> 2: .updatePosPL(Portfolio = pname, Symbol = as.character(symbol),
>>>> Dates = Dates, Prices = Prices, ... = ...)
>>>> 1: updatePortf(Portfolio = "p", Dates = paste("2009-01-01", "::",
>>>> "2009-04-06", sep = ""))
>>>>
>>>> I have looked in source code blotter / R / updatePortf.R and I see
>>>> the following lines:
>>>> startDate = first(xts:::.parseISO8601(Dates))$first.time-1 #does this
>>>> need to be a smaller/larger delta for millisecond data?
>>>> endDate = last(xts:::.parseISO8601(Dates))$last.time
>>>>
>>>> Apparently, the problem occurs in the "startDate" calculation. I tried
>>>> manually calling xts:::.parseISO8601('2009-04-06') and
>>>> xts:::.parseISO8601('::2009-04-06') and it works OK.
>>>>
>>>> Any ideas why xts:::.parse would be failing during script execution?
>>>>
>>>> --
>>>> Kind Regards,
>>>> Sergey Pisarenko.
>>>>
>>>> On Mon, Feb 27, 2012 at 6:51 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> The specific line this occurs at is:
>>>>>
>>>>> dummy <- updatePortf(Portfolio=portfolio, Dates=paste('::',
>>>>> as.Date(timestamp), sep=''))
>>>>>
>>>>> The function is inside of the sizing function. I have checked that
>>>>> timestamp is valid when this function is called so the problem has to
>>>>> do with some sort of a state problem in the portfolio.
>>>>>
>>>>> /Sergey
>>>>>
>>>>> On Mon, Feb 27, 2012 at 6:41 AM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Sorry for not following up sooner as I had been away for a week. I
>>>>>> have been able to further narrow down the problem. It occurs in the
>>>>>> sizing function and only when there are too many symbols. I have
>>>>>> further trimmed down the sample code.
>>>>>>
>>>>>> I will continue troubleshooting this function. Do you have any ideas
>>>>>> that would point me in the right direction?
>>>>>>
>>>>>> /Sergey
>>>>>>
>>>>>> On Fri, Feb 17, 2012 at 9:11 PM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>>>>> Garett,
>>>>>>>
>>>>>>> Thank you for the response and looking into this. The same is observed
>>>>>>> when I use MySQL (I keep a copy of data on a local DB). Yahoo does not
>>>>>>> have data for all symbols as well so this might explain some of the
>>>>>>> 404s.
>>>>>>>
>>>>>>> /Sergey
>>>>>>>
>>>>>>> On Fri, Feb 17, 2012 at 12:08 PM, G See <gsee000 at gmail.com> wrote:
>>>>>>>> I've been having trouble with yahoo over the last 24 hours or so.
>>>>>>>> Sometimes, it returns data; sometimes it doesn't
>>>>>>>>
>>>>>>>>> watched <- c("A", "AA", "AAN", "AAP", "AAT", "AAV")
>>>>>>>>> getSymbols(watched, src='yahoo', verbose=FALSE)
>>>>>>>> pausing 1 second between requests for more than 5 symbols
>>>>>>>> pausing 1 second between requests for more than 5 symbols
>>>>>>>> [1] "A" "AA" "AAN" "AAP" "AAT" "AAV"
>>>>>>>>> getSymbols(watched, src='yahoo', verbose=FALSE)
>>>>>>>> Error in download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m, :
>>>>>>>> cannot open URL
>>>>>>>> 'http://chart.yahoo.com/table.csv?s=AAT&a=0&b=01&c=2007&d=1&e=17&f=2012&g=d&q=q&y=0&z=AAT&x=.csv'
>>>>>>>> In addition: Warning message:
>>>>>>>> In download.file(paste(yahoo.URL, "s=", Symbols.name, "&a=", from.m, :
>>>>>>>> cannot open: HTTP status was '404 Not Found'
>>>>>>>>
>>>>>>>>
>>>>>>>> HTH,
>>>>>>>> Garrett
>>>>>>>>
>>>>>>>> On Fri, Feb 17, 2012 at 2:01 PM, Sergey Pisarenko <drseergio at gmail.com> wrote:
>>>>>>>>> Greetings to the R finance community,
>>>>>>>>>
>>>>>>>>> I have written a sizing function (pretty much a slight modification of
>>>>>>>>> a function from Guy Yollin's slides). The problem is that it fails
>>>>>>>>> whenever I use many symbols. It works fine for one or few symbols.
>>>>>>>>>
>>>>>>>>> The problem manifests itself inside of the sizing function, when
>>>>>>>>> updatePortf is run. The error I get is:
>>>>>>>>>> source('tainted_2.R')
>>>>>>>>> Error in if (nzchar(intervals[1])) s <- as.POSIXlt(do.call(firstof,
>>>>>>>>> parse.side(intervals[1]))) :
>>>>>>>>> argument is of length zero
>>>>>>>>>
>>>>>>>>> Traceback:
>>>>>>>>>> traceback()
>>>>>>>>> 13: xts:::.parseISO8601(Dates)
>>>>>>>>> 12: first(xts:::.parseISO8601(Dates))
>>>>>>>>> 11: .updatePosPL(Portfolio = pname, Symbol = as.character(symbol),
>>>>>>>>> Dates = Dates, Prices = Prices, ... = ...)
>>>>>>>>> 10: updatePortf(Portfolio = portfolio, Dates = paste("::", as.Date(timestamp),
>>>>>>>>> sep = "")) at tainted_2.R#100
>>>>>>>>> 9: osFUN(strategy = strategy, data = data, timestamp = timestamp,
>>>>>>>>> orderqty = orderqty, ordertype = ordertype, orderside = orderside,
>>>>>>>>> portfolio = portfolio, symbol = symbol, ... = ..., ruletype = ruletype,
>>>>>>>>> orderprice = as.numeric(orderprice))
>>>>>>>>> 8: function (data = mktdata, timestamp, sigcol, sigval, orderqty = 0,
>>>>>>>>> ordertype, orderside = NULL, threshold = NULL, tmult = FALSE,
>>>>>>>>> replace = TRUE, delay = 1e-04, osFUN = "osNoOp", pricemethod =
>>>>>>>>> c("market",
>>>>>>>>> "opside", "active"), portfolio, symbol, ..., ruletype,
>>>>>>>>> TxnFees = 0, prefer = NULL, sethold = FALSE)
>>>>>>>>> ...
>>>>>>>>> 7: do.call(fun, .formals)
>>>>>>>>> 6: ruleProc(strategy$rules[[type]], timestamp = timestamp, path.dep = path.dep,
>>>>>>>>> mktdata = mktdata, portfolio = portfolio, symbol = symbol,
>>>>>>>>> ruletype = type, mktinstr = mktinstr, ...)
>>>>>>>>> 5: applyRules(portfolio = portfolio, symbol = symbol, strategy = strategy,
>>>>>>>>> mktdata = mktdata, Dates = NULL, indicators = sret$indicators,
>>>>>>>>> signals = sret$signals, parameters = parameters, ..., path.dep = TRUE)
>>>>>>>>> 4: applyStrategy(strategy = "s", portfolios = "p")
>>>>>>>>> 3: eval.with.vis(expr, envir, enclos)
>>>>>>>>> 2: eval.with.vis(ei, envir)
>>>>>>>>> 1: source("tainted_2.R")
>>>>>>>>>
>>>>>>>>> The sizing function looks like:
>>>>>>>>>
>>>>>>>>> osPercentEquity <- function(timestamp, orderqty, portfolio, symbol,
>>>>>>>>> ruletype, ...) {
>>>>>>>>> posn <- getPosQty(portfolio, symbol, timestamp)
>>>>>>>>> if (posn == 0) {
>>>>>>>>> tempPortfolio <- getPortfolio(portfolio)
>>>>>>>>> dummy <- updatePortf(Portfolio=portfolio, Dates=paste('::',
>>>>>>>>> as.Date(timestamp), sep='')) # <<<<<< this is where it fails
>>>>>>>>> trading.pl <- sum(getPortfolio(portfolio)$summary$Net.Trading.PL)
>>>>>>>>> assign(paste("portfolio.", portfolio, sep=""), tempPortfolio, pos=.blotter)
>>>>>>>>> total.equity <- equity + trading.pl
>>>>>>>>> tradeSize <- total.equity * prcTrade
>>>>>>>>> ClosePrice <- as.numeric(Cl(mktdata[timestamp,]))
>>>>>>>>> orderqty <- sign(orderqty) * round(tradeSize / ClosePrice)
>>>>>>>>> return(orderqty)
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> The add.rule looks like:
>>>>>>>>> s <- add.rule(strategy=s, name='ruleSignal',
>>>>>>>>> arguments=list(data=quote(mktdata), sigcol='buySig', sigval=TRUE,
>>>>>>>>> orderqty=qtyDef, ordertype='market', orderside=NULL, threshold=NULL,
>>>>>>>>> osFUN='osPercentEquity', ruletype='enter'), type='enter')
>>>>>>>>>
>>>>>>>>> Traceback suggests that there is a problem with the date format. I
>>>>>>>>> have inserted print statements and verified that the date itself is
>>>>>>>>> fine. It is also weird that the issue does not occur when using few
>>>>>>>>> symbols.
>>>>>>>>>
>>>>>>>>> I have attached runnable code that reproduces the problem. Please
>>>>>>>>> excuse any apparent inefficiencies you see. I would like to get
>>>>>>>>> strategy working first and then optimize for performance.
>>>>>>>>>
>>>>>>>>> Is there something wrong with the approach I have taken? Once the
>>>>>>>>> number of symbols is reduced the problem does not trigger.
>>>>>>>>>
>>>>>>>>> Truly appreciate your help.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Kind Regards,
>>>>>>>>> Sergey Pisarenko.
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> R-SIG-Finance at r-project.org mailing list
>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>>>>>>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>>>>>>>> -- Also note that this is not the r-help list where general R questions should go.
More information about the R-SIG-Finance
mailing list