[R-SIG-Finance] [R-sig-finance] Preprocessing RData file (data.table and ff, bigmemory)
Jose Iparraguirre D'Elia
Jose at erini.ac.uk
Fri May 22 16:38:01 CEST 2009
You could also have a look at filehash, with which I've been playing around for a while.
Furthermore, I recently found a package still in Beta version, colbycol, written by Carlos J. Gil Bellosta (http://www.datanalytics.com) which seems to do what's on the tin: reading and managing large datasets well beyond the in-process memory limits...
From: r-sig-finance-bounces at stat.math.ethz.ch [mailto:r-sig-finance-bounces at stat.math.ethz.ch] On Behalf Of Steve Jaffe
Sent: 22 May 2009 15:19
To: r-sig-finance at stat.math.ethz.ch
Subject: Re: [R-SIG-Finance] [R-sig-finance] Preprocessing RData file (data.table and ff, bigmemory)
I'm new to R and interested in working with large amounts of data
(timeseries, but regularly spaced.) Can you point me to a good reference for
using data.table with bigmemory or ff?
(I'm a bit puzzled about what exactly these packages provide. As I
understand it, on 32-bit platforms files are subject to the same 2GB limit
as in-process memory, so I assume that dealing with a larger dataset still
requires breaking it up into multiple files...)
Thanks for your help.
I failed to point out that data.table can make use of both (?) those
packages. [ff, bigmemory]
It isn't a time-series library per se, but it make one very cool
in-memory database. Similar in spirit to some of the not-so-free ones
View this message in context: http://www.nabble.com/high-frequency-data-analysis-in-R-tp23654793p23671313.html
Sent from the Rmetrics mailing list archive at Nabble.com.
R-SIG-Finance at stat.math.ethz.ch mailing list
-- Subscriber-posting only.
-- If you want to post, subscribe first.
More information about the R-SIG-Finance