[R] memory management

[Prof Brian Ripley]

> Date: Fri, 10 Nov 2000 14:20:26 +0100 (CET)
> From: Roger Bivand <rsb at reclus.nhh.no>
> To: Pan_Yuming at aam.de
> cc: r-help at stat.math.ethz.ch
> Subject: Re: [R] memory management
> 
> On Fri, 10 Nov 2000 Pan_Yuming at aam.de wrote:
> 
> > read.table() is slower than scan(), but i wont use scan() if
> > read.table() can do it satifactoriely.  Kjetil gave me the right
> > direction. i tend to use
> >  a lot of loops in the program and that s not efficient.
> > 
> I have a feeling that there is an underlying issue concerning the
> treatment of character strings in read.table(), both for factors and as
> row names. A lot of con cells seem to be used up - that's where I've
> typically hit memory limits on reading largish files. If you can
> manage with scan(), and convert your character vectors to numeric (for
> later conversion to factor) before you read the file, you can use more
> memory for heap and less for con cells. If you are really stuck, then for
> special files, like images, it's best to write a small C function to suck
> in the data - this is much less challenging than it might seem, and gives
> you a chance to see how elegant R really is under the hood!

Or use package Rstreams on CRAN, thereby using someone else's C function.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._