[R-sig-Geo] Current options for creating/querying vector data WITHOUT loading them into memory?

[Barry Rowlingson]

Well, back when I wrote 'rmap' I abstracted out the storage of the
data from the data object... So your object in R could represent a
subset of a shapefile, and the code only grabbed that chunk of the
shapefile when it was needed, for example to plot (the R object was
basically the name of the shapefile plus a selection vector).

Then we threw that code out and sp classes were born!

 I've often thought about restoring some of this kind of
functionality, but R's object-oriented classes just frustrate me. Its
not so simple to build a superclass of sp class objects. Or maybe it
is now? For some value of 'simple'...

 Suppose you had a gigantic spatialite db - if you want to work with
it spatially (mapping, rgeos) you are going to have to get the bits
you need into main memory, so the simplest is just to load selections
into sp-class objects. Is that already possible with the OGR
spatialite driver? Can you also load subsets of shapefiles using some
SQL passed to the OGR shapefile driver?

 What would you want to do on whole-dataset objects of this class?
Would you want to do the processing on the database if possible (if
its PostGIS or Spatialite)? Or have an automatic chunking procedure
for operations that don't need the whole database at once, such as
finding centroids of polygons?

Hmmm thoughts thoughts thoughts and no action :( Sorry!

Barry



On Thu, Jan 16, 2014 at 6:52 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
> r-sig-geo'ers:
>
> As vector datasets are getting a lot larger, there is a limitation
> with the Spatial* formats in that they must be loaded into main
> memory.  I was curious what folks who have been dealing with massive
> vector files have come up with working within the R environment?  Has
> anyone played around with file geodatabases or spatialite formats (for
> instance)?  How are you creating/querying the data?
>
> Thanks!
>
> --j
>
> --
> Jonathan A. Greenberg, PhD
> Assistant Professor
> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
> Department of Geography and Geographic Information Science
> University of Illinois at Urbana-Champaign
> 259 Computing Applications Building, MC-150
> 605 East Springfield Avenue
> Champaign, IL  61820-6371
> Phone: 217-300-1924
> http://www.geog.illinois.edu/~jgrn/
> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo