[R-SIG-Finance] xts + indexing for tick data?

[Andrew Piskorski]

On Thu, Aug 05, 2010 at 02:09:45PM -0500, Jeff Ryan wrote:

> The general design is something like a data.frame, but implemented as
> an environment.  The internal structure shouldn't be a concern though,
> just note that the interface attempts to behave like a data.frame.

Is this interface available in the indexing package, or will it be?

> In essence, you can take any type of data that mmap can use and put it
> into this indexed environment/db.

> Many many GB or bigger is easy even on a laptop.

Cool.  Is there an example somewhere of how I should take, say, a
simple data frame, design an appropriate set of column files and
indexes, and then populate them with tons of data?  Should I just go
ahead and design my own little layout scheme, or would I be needlessly
reinventing the wheel?

> http://www.insightalgo.com/indexing_mmap.pdf

Nice option contracts use case.  For populating your 12 GB of binary
data files, did you:

1. Do it with R.
2. Use some of the C code in your mmap or indexing packages to help
   you, but without R.
3. Do it using some other entirely unrelated code or tool.

I'm curious about which approach you found most useful for data
loading and maintenance, particularly while making sure that the
various different files for each of your 13 columns are kept in sync.
I've got a similar amount of futures tick data, currently in a totally
unwieldy vendor format, and need to decide how to arrange it...

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/