[R-sig-Geo] reading ascii grid memory issues

Roger Bivand Roger.Bivand at nhh.no
Fri Dec 5 08:47:59 CET 2008

On Fri, 5 Dec 2008, Alexander.Herr at csiro.au wrote:

> Thanks Barry
>> Hi List,
>> I am unable to read in a 7.8Gb ascii grid using readAsciiGrid 
>> {maptools}/ read.asciigrid {sp}; R runs out of memory. I have 4Gb ram 
>> and 4Gb swap, so things are getting tight.
>> I am wondering if anyone has alternative options (preferably with 
>> example) that enable to read in large grids, do some calculations and 
>> save a new grid?

This is another case of posting on r-help and R-sig-geo, leading to the 
thread getting confused. I replied on r-help:


saying that using tiling with rgdal functions - which have a driver for 
this data - is the obvious way to go, and that the raster package under 
development on R-forge looks very promising for letting users automate the 
process of reading rasters in tiles.

See also the relevant thread here:



> Depends on what you want to do with it. What's the dimension of the grid in rows x columns?
> -->dimensions 80264x73772 for the largest grid and 28848x47030 for current grid
> Firstly, can you convert it to a more efficient format? Ascii Grid files 
> are generally long lines of numbers, looking like 2.333535 4.457574 
> 2.332336 5.445775 and so on. Maybe 10 characters per grid square. You 
> might be able to squeeze it to 4 characters per grid square by storing 
> as raw floating point numbers.
> -->yes, depends on what formats R is able to read in
> Next, can you just process it serially, one line or grid square at a 
> time? Do you need to store it all in memory at the same time?
> --> I can process serially, I don't need to store it in memory for the 
> simple math calculations, only for subsequent analysis. But once 
> calculations are done I can resample/aggregate to higher resolution.

Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no

More information about the R-sig-Geo mailing list