[R-sig-Geo] Replacement shapefiles with SpatiaLite for R-geo packages(spdep, rgeos etc.)

Mon Dec 3 20:07:37 CET 2012

On Mon, 3 Dec 2012, Roger Bivand wrote:

> On Sun, 2 Dec 2012, Edzer Pebesma wrote:
>
>> Roger, what do you mean by "Its [Spatialite's] future status is unknown"?

In addition, in trying to see the benefits of the file format, I loaded:

https://groups.google.com/group/spatialite-users/attach/9fd2f47b1a686829/meridian_test.7z?part=4&view=1

(3.6 MB) into R as three layers (rivers, rail, woods). In R, the sum of 
the layers is 20KB. Am I missing something, or is the format bloated? 
Maybe the choice made by GADM to distribute in RData format makes more 
sense? GDAL already supports SpatialGridDataFrame objects.

Roger

>
> Only that people have been talking it up, but I haven't seen data on adoption 
> (or repeated pleas from users) that would show that it has reached a critical 
> mass of any kind, let alone of the mass/mess of shapefiles. So for the time 
> being, OGR is the obvious route, but building spatialite without Excel, 
> without GEOS, etc. I found its compile option to include GEOS development 
> (trunk) features very concerning - even GEOS advise against building against 
> its experimental features.
>
>> 
>> A few months ago I found an interesting discussion here:
>> https://groups.google.com/forum/?hl=en&fromgroups=#!topic/geospatial-mobile-data-format-for-vectors/VEri8doLNz4
>> which convinced me that spatialite is a software, rather than an open
>> (file) format - the format is of course there, but not specified
>> otherwise than by the software that reads/writes it (sqlite).
>> 
>
> Well, if it is software, someone could Rcpp it, but I don't see why, when 
> what it offers for us is GEOS again, and probably more muddle on CRS and 
> codepages - controlling what GDAL/OGR do is a serious enough task without 
> mixing in less well-judged and cautious software (QGIS?). My take on the 
> discussion you cite would be close to Paul Ramsey's doubts.
>
>> I'm also interested in spatialite, as I do not see other really useful
>> data formats (although it's not a format!) that can be used to exchange
>> spatio-temporal data to other environments, say ArcGIS or so.
>> 
>
> Well, OGR could move in that direction, couldn't it? Then some drivers would 
> be have temporal capabilities, others not; of course, someone would need to 
> do some work on generalisable data models, but OGR could lead here (INPE?).
>
>> Taking the OGR path is of interest, but of course OGR has no idea of
>> time. For this path, the main burden would be there where people prepare
>> the distributions that are used by the windows and OS-X CRAN build
>> trains, right?
>
> That's why contributing upstream to OGR is a better idea, unless the data 
> models would become too convoluted. The same applies - as I wrote - to SQL in 
> OGR.
>
>> 
>> The R SQLiteMap package, IIRC, was built on top of the SQLite driver
>> directly without OGR, and parsed the geometry columns directly; I once
>> looked at it, but was somehow not overwhelmed by enthousiasm and forgot
>> why -- was it restricted to WKT representation of geometry?
>> 
>
> It was written for a particular purpose some time ago, and is now orphaned by 
> its authors. CRAN policy is to orphan maintainerless packages that have 
> unanswered issues.
>
> Best wishes,
>
> Roger
>
>> Best regards,
>> 
>> 
>> On 12/02/2012 03:40 PM, Roger Bivand wrote:
>>> On Sun, 2 Dec 2012, hi_ono2001 at ybb.ne.jp wrote:
>>> 
>>>> Hello.
>>>> 
>>>> Recently SpatiaLite 4.0 was released. It has lots of powerful, fast
>>>> and robust functions for Geoprocessing(spatial queries, overlay
>>>> analysis, creating topology, network analysis, gridding, creating
>>>> TIN/voronois). Some people say SpatiaLite is one of candidate to next
>>>> standard spatial data format.
>>> 
>>> Spatialite is supported by OGR in rgdal if the drivers are available.
>>> Its future status is unknown, but of course we should try to make life
>>> easier for its users. If you could document how easy it is to include
>>> the driver in the CRAN Windows and OSX binaries, that would help. In my
>>> opinion, Spatialite should be used through OGR in R, and should not be
>>> used for topological operations, which blunt the sense of having shared
>>> classes in R for spatial data. Using OGR means that things stay
>>> maintainable.
>>> 
>>>> 
>>>> Current rgeos package seems to be slower and unstable for bigger
>>>> spatial data. And I think if using SpatiaLite, spdep could create
>>>> spatial weight matrices more speedy than current ones. If this, R will
>>>> be exactly "R as a GIS".
>>> 
>>> Since Spatialite uses the same GEOS dependency as rgeos, you should
>>> substaniate this claim with clear reproducible cases; rgeos is now
>>> pretty stable, and misbehaviour now is usually because of erroneous
>>> topologies (overlaps, slivers and dangles). I would strongly advise
>>> against thinking that exporting to Spatialite then GEOS, doing GEOS
>>> there, and importing again, will be faster or more stable than
>>> converting Spatial* objects to GEOS objects, doing GEOS, and converting
>>> back again - the steps are very similar, and such a claim would need to
>>> be studied carefully. The same time could rather be used to contribute
>>> to rgeos. Spatial weights matrices created in spdep can use STRtrees
>>> from GEOS, and are fast (90K polygons in 10 seconds). But this isn't the
>>> default, you need to use extra arguments.
>>> 
>>>> 
>>>> Unfortunately SQLiteMap, R package for accessing SpatiaLite, has been
>>>> removed from CRAN since this September.
>>> 
>>> CRAN has been making its management more systematic - if maintainers do
>>> not respond to requests for corrections, the source packages are
>>> archived until the maintainers comply. This is why maintainability is
>>> crucial. Arguably, the package would also be redundant if OGR Spatialite
>>> drivers were available for those dependent on the CRAN rgdal binaries.
>>> 
>>>> 
>>>> Any plan to employing  this SpatiaLite for current R-geo packages.
>>>> 
>>> 
>>> Please contribute a vignette to a suitable package, showing how one can
>>> use this storage format with R already. For use of SQL, rather consider
>>> contributing to helping add OGR SQL to readOGR in rgdal, then other
>>> drivers would benefit too.
>>> 
>>> Best wishes,
>>> 
>>> Roger
>>> 
>>>> Thanks in advance.
>>>> 
>>>> 
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>> 
>>> 
>> 
>> 
>
>

-- 
Roger Bivand
Department of Economics, NHH Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no