[R] Large data sets in R

[Robert Gentleman]

As part of a new project I am working on we have a preliminary version
of an HDF5 library available. This will allow you to work with very
large datasets. We are hoping for a release in the next couple of
weeks but if you have some experience with hdf5 and want to look at an
early version please let me know.

   Robert

On Mon, Oct 29, 2001 at 09:20:52AM -0500, Liaw, Andy wrote:
> Can this be added to the official FAQ (or perhaps a more generic response)?
> I believe the question qualifies as a legitimate FAQ.
> 
> (Or maybe it's there already?  Haven't check it lately...)
> 
> Andy
> 
> > -----Original Message-----
> > From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk]
> > Sent: Monday, October 29, 2001 8:45 AM
> > To: Laura Gross
> > Cc: r-help at stat.math.ethz.ch
> > Subject: Re: [R] Large data sets in R
> > 
> > 
> > On Mon, 29 Oct 2001, Laura Gross wrote:
> > 
> > > As a new user of R, I'm wondering what the maximum matrix 
> > size is in R?
> > > I have a large data set consisting of 9000 people and want 
> > to be able to
> > > create large matrices involving these. Is R suitable for large data
> > > sets?
> > 
> > The answer depends on
> > 
> > 1) Your platform
> > 2) The amount of memory you have (especially of RAM)
> > 3) What you want to do with 9000 cases, in particular how 
> > much information
> > you have on them.
> > 
> > As a general guide, I would say 9000 cases was not large for R on a
> > machine with say 256Mb RAM.  But a cluster analysis needing a
> > 9000*9000 matrix would be too large.
> > 
> > -- 
> > Brian D. Ripley,                  ripley at stats.ox.ac.uk
> > Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> > University of Oxford,             Tel:  +44 1865 272861 (self)
> > 1 South Parks Road,                     +44 1865 272860 (secr)
> > Oxford OX1 3TG, UK                Fax:  +44 1865 272595
> > 
> > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
> > -.-.-.-.-.-.-.-.-
> > r-help mailing list -- Read 
> > http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> > Send "info", "help", or "[un]subscribe"
> > (in the "body", not the subject !)  To: 
> > r-help-request at stat.math.ethz.ch
> > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
> > _._._._._._._._._
> > 
> 
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

-- 
+---------------------------------------------------------------------------+
| Robert Gentleman                 phone : (617) 632-5250                   |
| Associate Professor              fax:   (617)  632-2444                   |
| Department of Biostatistics      office: M1B28
| Harvard School of Public Health  email: rgentlem at jimmy.dfci.harvard.edu   |
+---------------------------------------------------------------------------+
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._