[R] Huge data frames?

[ripley@stats.ox.ac.uk]

RODBC definitely uses more memory than read.table.

One day I'll rewrite RODBC to use the more efficient internals of the
current read.table.

On Wed, 28 Aug 2002 chr.schulz at email.de wrote:

> ..perhaps you have more access
> when you use the Rodbc Package and MySQL as
> Data-Store !?
>
> channel  <- odbcConnect("db_name","login","pass")
> data  <- sqlFetch(channel, data, errors = TRUE, as = "data frame", nullstring = "sysmis", na.strings = "NA")
>
> ..or sqlQuery !
>
>
> good luck ;-)
>
> regards,christian
>
>
>
> Magnus Lie Hetland <magnus at hetland.org> schrieb am 28.08.02 07:44:33:
> > A friend of mine recently mentioned that he had painlessly imported a
> > data file with 8 columns and 500,000 rows into matlab. When I tried
> > the same thing in R (both Unix and Windows variants) I had little
> > success. The Windows version hung for a very long time, until I
> > eventually more or less ran out of virtual memory; I tried to set the
> > proper memory allocations for the Unix version, but it never seemed
> > satisfied :]
> >
> > I used read.table -- should I have used something else? Is it even
> > possible to work with this large files? I assume a memory-mapped
> > binary file would have been quite efficient (as opposed to an
> > in-memory parsed text file) -- is something like that even possible in
> > R?
> >
> > --
> > Magnus Lie Hetland                                  The Anygui Project
> > http://hetland.org                                  http://anygui.org
> > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> > r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> > Send "info", "help", or "[un]subscribe"
> > (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
>
>
>
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._