[Rd] R's IO speed
Prof Brian Ripley
ripley at stats.ox.ac.uk
Mon Jan 3 22:20:07 CET 2005
Use memory.size(max=TRUE) in Windows, or some other profiling memory
On Mon, 3 Jan 2005, Vadim Ogranovich wrote:
> A technical question here: how does one measure the memory overhead
> mentioned below? I have a set of functions of my own and would like to
> profile them.
>> -----Original Message-----
>> From: r-devel-bounces at stat.math.ethz.ch
>> [mailto:r-devel-bounces at stat.math.ethz.ch] On Behalf Of Prof
>> Brian Ripley
>> Sent: Sunday, December 26, 2004 2:04 AM
>> To: R-devel at r-project.org
>> Subject: [Rd] R's IO speed
>> R-devel now has some improved versions of read.table and write.table.
>> For a million-row data frame containing one number, one
>> factor with few levels and one logical column, a 56Mb object.
>> generating it takes 4.5 secs.
>> calling summary() on it takes 2.2 secs.
>> writing it takes 8 secs and an additional 10Mb.
>> saving it in .rda format takes 4 secs.
>> reading it naively takes 28 secs and an additional 240Mb
>> reading it carefully (using nrows, colClasses and
>> comment.char) takes 16 secs and an additional 150Mb (56Mb of
>> which is for the object read in).
>> (The overhead of read.table over scan was about 2 secs,
>> mainly in the conversion back to a factor.)
>> loading from .rda format takes 3.4 secs.
>> [R 2.0.1 read in 23 secs using an additional 210Mb, and wrote
>> in 50 secs using an additional 450Mb.]
>> Will Frank Harrell or someone else please explain to me a
>> real application
>> in which this is not fast enough?
>> Brian D. Ripley, ripley at stats.ox.ac.uk
>> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
>> University of Oxford, Tel: +44 1865 272861 (self)
>> 1 South Parks Road, +44 1865 272866 (PA)
>> Oxford OX1 3TG, UK Fax: +44 1865 272595
>> R-devel at stat.math.ethz.ch mailing list
Brian D. Ripley, ripley at stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
More information about the R-devel