[R] R for large data sets

[Ernesto Jardim]

Hi

I'm using some large datasets and I found the ROracle package to be of
great help.

If you have the chance to create a database in Oracle or MySQL with one
single table for your dataset, you can then use the ROracle package to
access the dataset. I found several advantages on that. 

I don't import the data into my environment. I use a small function (see
below) to access the dataset and because the result is a data.frame you
can use it as usually.

Your environment will not be to large and you'll have the ram memory
less full.

It's easier to select subsets with SQL than S/R language. 

Hope it helps

Regards

EJ

--//--

ora.fun <- function(){

        library(ROracle)
        m <- dbManager("Oracle")
        con <- dbConnect(m,user="user",password="password")
        dat <- quickSQL(con,"select ...")
        close(con)
        unload(m)
        dat

}

--//--

On Tue, 2002-01-15 at 19:43, Prof Brian Ripley wrote:
> On Tue, 15 Jan 2002, wei, xiaoyan wrote:
> 
> > As a part of our regular data analysis, I have to read in large data sets
> > with six columns and about a million rows. In Splus, this usually take a
> > couple of minutes. I just tried R, it seems take forever to use read.table()
> > to read in the data frame! It did not help much even though I specified
> > colClasses and nrows in read.table().
> >
> > How is R's ability to analyze large data sets? I used R on solaris 2.6 and I
> > used all default compilation flags when building the R package. Will it help
> > if I use some compilation flags with higher optimization level?
> 
> It will help to use R-patched, since I guess you are using 1.4.0.
> Also, look in the list archives, as I answered this more fully earlier
> today.
> 
> In either S-PLUS or R, scan would be a better choice for such a dataset.
> 
> -- 
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272860 (secr)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
> 
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._