[Rd] memory problem read.table v array (PR#9526)
Prof Brian Ripley
ripley at stats.ox.ac.uk
Fri Feb 23 00:18:18 CET 2007
There is nothing to reproduce here: we do not have 639.txt.
But note that read.table returns a data frame, and ?array has
data: a vector (including a list) giving data to fill the array.
so I do wonder if this is what you intended: you seem to have tried to
create an array list with 639*639 elements. It is certainly not the same
sort of object as array(0,dim=c(409600,1)).
Had 639.txt contained 639 rows of reals, your 'a' would be about
639*639*639*8 bytes, beyond the address space of your machine.
as.matrix(read.table("639.txt", header=FALSE)) might have been what you
are looking for.
On Thu, 22 Feb 2007, wlangdon at essex.ac.uk wrote:
> Full_Name: bill langdon
> Version: 2.4.1
> OS: ubuntu
> Submission from: (NULL) (155.245.58.159)
>
>
> #WBL 22 Feb 2007 ubuntu
> R.version
> #platform i486-pc-linux-gnu
> #arch i486
> #os linux-gnu
> #system i486, linux-gnu
> #status
> #major 2
> #minor 4.1
> #year 2006
> #month 12
> #day 18
> #svn rev 40228
> #language R
> #version.string R version 2.4.1 (2006-12-18)
>
> #if matrix "a" is created by array "vals" is created ok
>
> #if matrix "a" is created by read.table,
> #peak resource use (CPU, memory) by "array()" is excessive
>
> #a = array(0,dim=c(409600,1)); #ok
> #a = read.table("big.txt",header=FALSE); #all memory used
> #a = read.table("639x639.txt",header=FALSE); #all memory used
> #a = read.table("4096.txt",header=FALSE); #all memory used
> #a = read.table("4096nocomment.txt",header=FALSE); #all memory used
> #a = read.table("tiny.txt",header=FALSE); #ten lines ok
> #a = read.table("1000.txt",header=FALSE); #all memory used
> a = read.table("639.txt",header=FALSE); #uses 1.6047029GB
> dim(a)
> dd = 639;
> vals = array(a,dim=c(dd,dd));
--
Brian D. Ripley, ripley at stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
More information about the R-devel
mailing list