[R-sig-Geo] memory issue on R with Linux 64

[Robert Hijmans]

Herry,

This is how you can do it in package{raster}. (revision 209;  a build
should be available within 24 hours).

Following Edzer's example:

require(raster)
library(maptools)
# read a SpatialPolygonsDataFrame
nc <- readShapePoly(system.file("shapes/sids.shp",
      package="maptools")[1], proj4string=CRS("+proj=longlat +datum=NAD27"))
# create a new RasterLayer object from the polygon bounding box and
set row / cols.
rs <- rasterFromBbox(nc, nrows=54, ncols=176)
# transfer polygons to raster, use values of column 13 in the polygon dataframe
rs <- polygonsToRaster(nc, rs, field=13)
# plot, either directly
plot(rs)
plot(nc, add=T, border="blue")
# or via sp for a real map
x11()
spplot(asSpGrid(rs), col.regions=bpy.colors())

also see the example in ?polygonsToRaster

The polygonsToRaster function works for very large rasters (row by row
processing if you provide an output file name). I have only tested if
for a very limited number of very simple cases, so user beware...
The algorithm needs optimization for speed, so that might be a problem
for your very large grids. (particularly of your polygons are also
complex). It also needs some tweaking (and options) for deciding when
a polygon is in a grid cell. As is, the intention is that a polygon
has to overlap with the center of a cell to be considered inside.

Robert




On Thu, Jan 29, 2009 at 3:09 PM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
> On Thu, 29 Jan 2009, Alexander.Herr at csiro.au wrote:
>
>>
>> Hi List,
>>
>> I get an error using readGDAL{rgdal}: cannot allocate vector of size 3.1
>> Gb
>
> This is a tile of your 73K by 80K raster, right? One possibility is to use
> smaller tiles, another to get more memory (as Edzer wrote), a third to use
> lower level functions in rgdal to avoid duplication (and repeated gc()) - in
> readGDAL the data read by getRasterData() are copied, so at least doubling
> memory usage.
>
> Do you need to read the raster? If this is the overlay problem, you should
> be able to use the output of GDALinfo for your raster to build a
> GridTopology and SpatialGrid, and overlay (tiles of) that on the
> SpatialPolygons (tiles of that because overlay() will generate cell centre
> coordinates and do point in polygon, so you're still stuck with too many
> coordinates). The next issue would be to copy out the polygon IDs, or the
> extracted values, as a raster - here the forthcoming raster package on
> R-Forge may be what you need.
>
> Roger
>
>
>>
>> I am using Linux 64bit (opensuse 11) with 4 gig swap and 4 gig Ram and R
>> 2.8.0.
>>
>> The load monitor shows that most of Ram is used up and then when Swap use
>> starts increasing, R returns the error.
>>
>> Is there anything I should do within R to circumvent this?
>>
>>
>> Any help appreciated
>> Thanks
>> Herry
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at stat.math.ethz.ch
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> --
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian School of
> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
> e-mail: Roger.Bivand at nhh.no
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>