[R-sig-Geo] where to tweak memory allocation settings

Jochen Albrecht jochen at hunter.cuny.edu
Thu Oct 8 17:37:13 CEST 2009


Fellow spatial statisticians:
I am trying to align a bunch of world-wide datasets at 0.1 degree 
resolution. I figured that the easiest way to do that is to create a 
dataset where the rownames form the index that allows to relate all my 
variable-sized point datasets to. I therefore created a shapefile base 
dataset that contains appr. 6.5 million points and had hoped that I 
could read this into R to then link all my other data to. Unfortunately, 
I ran into memory allocation problems (see error messages attached to 
this email).
My hunch is that these can be overcome by just changing some settings 
(hopefully without having to recompile R sources). Do you have 
suggestions what these are and what the practically hard limit is? 6.5 
million points is large but not uncommon these days, so I figured that 
this should be doable without embarking on major efforts.
Alternatively, would it save me a lot of memory space if I tried to read 
this into a SpatialPixel or SpatialGrid structure?
Cheers,
     Jochen

 > basepoints = readOGR(".", "basepoints")
OGR data source with driver: ESRI Shapefile
Source: ".", layer: "basepoints"
with  3465355  rows and  7  columns
Feature type: wkbPoint with 2 dimensions
Warning in data.frame(dlist) :
  Reached total allocation of 1535Mb: see help(memory.size)
Warning in data.frame(dlist) :
  Reached total allocation of 1535Mb: see help(memory.size)
Warning in data.frame(dlist) :
  Reached total allocation of 1535Mb: see help(memory.size)
Warning in data.frame(dlist) :
  Reached total allocation of 1535Mb: see help(memory.size)
Warning in as.data.frame.integer(x[[i]], optional = TRUE) :
  Reached total allocation of 1535Mb: see help(memory.size)
Warning in as.data.frame.integer(x[[i]], optional = TRUE) :
  Reached total allocation of 1535Mb: see help(memory.size)
Warning in as.data.frame.integer(x[[i]], optional = TRUE) :
  Reached total allocation of 1535Mb: see help(memory.size)
Warning in as.data.frame.integer(x[[i]], optional = TRUE) :
  Reached total allocation of 1535Mb: see help(memory.size)
Error: cannot allocate vector of size 13.2 Mb



More information about the R-sig-Geo mailing list