[R-SIG-Finance] In highfrequency package `convert` function creates folder structure, but no .RData file with TickData.com data

[Joshua Ulrich]

Nick,

On Thu, Sep 18, 2014 at 12:41 AM, Nick White <n-e-w at qtradr.net> wrote:
> Hi all,
>
> I also raised the following (summarised, but pretty much verbatim) as an
> issue on the highfrequency package's github Issues page... but that doesn't
> appear to get a lot of attention:
>
> https://github.com/jonathancornelissen/highfrequency/issues/3
>
I think the github account was only used for the initial Google Summer
of Code project.  The current code base seems to be on R-Forge:
https://r-forge.r-project.org/projects/highfrequency/

> Here's the same substance:
>
> I am new to using the package. The following issue is with TickData.com
> files.
>
> I have setup the folders with a single day's worth of data for XYZ as per
> the documentation:
>
> from = "2014-09-02";
> to     = "2014-09-02";
> datasource = "~/raw_data";
> datadestination = "~/xts_data"
>
> in ~/raw_data I have placed the files:
>
> XYZ_quotes.asc
> XYZ_trades.asc
>
> which are simply copied and renamed files from my original TickData folder.
> Note, I have *not* extracted these using TickWrite7. These are just the raw
> .asc files from the download directory.
>
> I then run the convert function as follows:
>
>> convert(from=from, to=to, datasource=datasource, datadestination=datadestination, trades = TRUE,quotes = TRUE, ticker="XYZ", dir = TRUE, format = "%d/%m/%Y %H:%M:%OS",extension = "tickdatacom", header = TRUE,  onefile = TRUE )
>
> which begins to run and take some time....but then produces this error:
>
> Error in if (length(c(year, month, day, hour, min, sec)) == 6 && c(year,  :
>   missing value where TRUE/FALSE needed
> In addition: Warning messages:
> 1: In as_numeric(YYYY) : NAs introduced by coercion
> 2: In as_numeric(YYYY) : NAs introduced by coercion
>
> which, when calling traceback produces:
>
>> traceback()
> 7: (function (year = 1970, month = 12, day = 31, hour = 23, min = 59,
>        sec = 59, subsec = 0.99999, tz = "")
>    {
>        if (!missing(sec) && sec%%1 != 0)
>            subsec <- 0
>        sec <- ifelse(year < 1970, sec, sec + subsec)
>        mon.lengths <- c(31, 28, 31, 30, 31, 30, 31, 31, 30, 31,
>            30, 31)
>        if (missing(day)) {
>            day <- ifelse(month %in% 2, ifelse(((year%%4 %in% 0 &
>                !year%%100 %in% 0) | (year%%400 %in% 0)), 29, 28),
>                mon.lengths[month])
>        }
>        if (length(c(year, month, day, hour, min, sec)) == 6 && c(year,
>            month, day, hour, min, sec) == c(1969, 12, 31, 23, 59,
>            59) && Sys.getenv("TZ") %in% c("", "GMT", "UTC"))
>            sec <- sec - 1
>        ISOdatetime(year, month, day, hour, min, sec, tz)
>    })(year = NA_real_, tz = "GMT")
> 6: do.call(lastof, parse.side(intervals[2], intervals[1]))
> 5: as.POSIXlt(do.call(lastof, parse.side(intervals[2], intervals[1])))
> 4: .parseISO8601(ii, .index(x)[1], .index(x)[nr], tz = tz)
> 3: `[.xts`(alldata, as.character(dates[i]))
> 2: alldata[as.character(dates[i])]
> 1: convert(from = from, to = to, datasource = datasource,
> datadestination = datadestination,
>        trades = TRUE, quotes = TRUE, ticker = "XYZ", dir = TRUE,
>        format = "%d/%m/%Y %H:%M:%OS", extension = "tickdatacom",
>        header = TRUE, onefile = TRUE)
>
It's really hard to say without a reproducible example, but the error
occurs either on line 2301 or 2308 of realized.R.  And dates is
defined as:

dates = unique(as.Date(index(alldata)))

which makes me think this is a TZ issue.  The default TZ for as.Date
is "UTC", but the TZ of alldata is likely different.  You could cross
your fingers and run Sys.setenv(TZ="UTC") at the beginning of your R
session and see if that works...

>
> This occurs using extracted .asc files from TickWrite7 both where Date and
> Time field is chosen or where individual Date and Time fields were selected.
>
>
> This also happens whether I extract the files using TW7 or just get use the
> raw files in my TickWrite data download folders as noted above.
>
>
> This is on R3.0.2, x64, Win8.1P, highfrequency v = 0.3
>
>
> Any help v much appreciated.
>
>
> Cheers,
>
> n.
>

Best,
--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com