[R-sig-Geo] rgdal 1.1-1 bug?

Tue Dec 22 11:43:40 CET 2015

On Mon, 7 Dec 2015, Worrall, James -FS wrote:

> Thank you Tom and Roger for your attention to this issue.
>
> Here is a link to a zipfile (17 MB).  It contains two 
> folders/shapefiles.  One is the single big polygon with many holes that 
> I had sent to Tom.  The other is the shapefile it was extracted from, 
> with many additional polygons showing similar problems.
>
> https://www.dropbox.com/s/xkp8sdoccg6jyg4/offending_shapefile.zip?dl=0
>
> These shapefiles are from Arc and have never been written by writeOGR, 
> so should be fine when first opened.

Using these files, it was possible to track and fix the bug, introduced in 
release 1.1-1 while fixing a different problem. Thanks to Jim and Tom for 
assistance and patience. Version 1.1-3 contains the fix and has been 
released on CRAN (source and Windows binary packages). OSX users needing 
the new version will have to install from source until the CRAN OSX build 
train catches up.

Also thanks to Brian Ripley and Uwe Ligges, the CRAN Windows binary is now 
built using GDAL 2.0.1, and also includes SQLite-based drivers.

Roger

>
> For now, I have been unable to revert to the earlier version of rgdal as 
> R complains that it doesn't match the R version.

> Jim
>
> -----Original Message-----
> From: R-sig-Geo [mailto:r-sig-geo-bounces at r-project.org] On Behalf Of Roger Bivand
> Sent: Saturday, December 05, 2015 3:50 AM
> To: Tom Philippi
> Cc: r-sig-geo at r-project.org
> Subject: Re: [R-sig-Geo] rgdal 1.1-1 bug?
>
> On Sat, 5 Dec 2015, Tom Philippi wrote:
>
>> James & all--
>
> If you make the offending shapefile available, I'll look at this in a week or so (no time before then). The possible change is from:
>
> https://stat.ethz.ch/pipermail/r-sig-geo/2015-November/023656.html
>
> and affected writing - before Shapefiles were written as-is, from 1.1-1 all output geometries without comments are put through
> OGRGeometryFactory::organizePolygons() in GDAL/OGR. This may be a case in which the OGR function fails to organise the polygons in the intended fashion. I have a build of rgdal 1.1-2 for Windows using GDAL
> 2.0.1 (thanks again to Brian Ripley) which you may try, but it will most likely not change anything. Also, we don't know whether the input data (Polygons objects) have comments or not, but if read through readOGR(), the Shapefile driver also runs OGRGeometryFactory::organizePolygons() on read.
>
> I need access to the offending shapefile to see whether alternative settings to OGRGeometryFactory::organizePolygons(), or a user option to avoid this very sensible function with possibly bad data, might alleviate the problem. Please put on URL or attach offlist.
>
> Roger
>
>> With your shapefile I can reproduce your problem under Windows 7x64 in
>> R
>> 3.2.2 rgdal_1.1-1 sp_1.2-1, but not R 3.1.2 rgdal_0.9-3 sp_1.1-1 or
>> earlier.  Both rgdals are linked to gdal 1.11.2 released 2015/02/10.
>> I also can reproduce the issue writing to & reading from GeoJSON
>> instead of shapefiles.  Files written by rgdal 1.1-1 have the lines
>> connecting holes when read by rgdal_0.9-3, but files written by
>> rgdal_0.9-3 are read fine by 1.1-1.
>>
>> The rarity about the shapefile in question is that it has 57 holes in
>> the same polygon, which (along with the lines connecting holes)
>> suggests that the issue might be related to SpatialPolygons not
>> assigning holes to specific (non-hole) polygons, see:
>> http://finzi.psych.upenn.edu/R/library/rgeos/html/comment-functions.ht
>> ml
>>
>> I don't have time (or ability) to go much further right now in terms
>> of finding the change, but perhaps someone else can?
>>
>> Tom 2
>>
>> On Fri, Dec 4, 2015 at 2:34 PM, Michael Sumner <mdsumner at gmail.com> wrote:
>>
>>>
>>>
>>> On Sat, 5 Dec 2015 at 05:26 Worrall, James -FS <jworrall at fs.fed.us> wrote:
>>>
>>>> [image: image001.gif]
>>>>
>>>> We have been happily using rgdal to read and write ESRI Shapefiles.
>>>> Recently my colleague began using it.  We noticed that after saving
>>>> and reopening files, many polygons had been reconfigured, as if the
>>>> order of the vertices had become jumbled and there were many
>>>> straight lines cutting across the polygon.  I’m not sure if an image
>>>> can be pasted in this list but I’ll try.
>>>>
>>>> I did the same thing on my computer and the problem did not occur.
>>>> We found that I had an older version of rgdal, 1.0-7.  When I
>>>> updated it to 1.1-1, the problem occurred just as it had with her.
>>>>
>>>> Simply opening and saving a shapefile causes the problem and
>>>> essentially jumbles the shapefile.  The bug, if it is one, was
>>>> introduced after 1.0-7
>>>>
>>>
>>>
>>>
>>>
>>> I can't reproduce this. Can you provide a reproducible example?
>>> Also, "opening and saving a shapefiles" is not the way to describe
>>> what is done in the usual rgdal workflow, so it's not clear what you
>>> did. Here's what I
>>> tried:
>>>
>>> library(maptools)
>>> data(wrld_simpl)
>>>
>>> library(rgdal)
>>> #rgdal: version: 1.1-1, (SVN revision 572) # Geospatial Data
>>> Abstraction Library extensions to R successfully loaded # Loaded GDAL
>>> runtime: GDAL 1.11.2, released 2015/02/10 # Path to GDAL shared
>>> files: E:/inst/R/R/library/rgdal/gdal # GDAL does not use iconv for
>>> recoding strings.
>>> # Loaded PROJ.4 runtime: Rel. 4.9.1, 04 March 2015, [PJ_VERSION: 491]
>>> # Path to PROJ.4 shared files: E:/inst/R/R/library/rgdal/proj #
>>> Linking to sp version: 1.2-1
>>>
>>>
>>> writeOGR(wrld_simpl, ".", "1.1-1", "ESRI Shapefile")
>>> x1 <- readOGR(".", "1.1-1")
>>> plot(x1)  ## all good
>>>
>>>
>>>  sessionInfo()
>>> R version 3.2.2 Patched (2015-08-27 r69201)
>>> Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 8.1
>>> x64 (build 9600)
>>>
>>> locale:
>>> [1] LC_COLLATE=English_Australia.1252
>>> LC_CTYPE=English_Australia.1252 [3]
>>> LC_MONETARY=English_Australia.1252 LC_NUMERIC=C [5]
>>> LC_TIME=English_Australia.1252
>>>
>>> attached base packages:
>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>
>>> other attached packages:
>>> [1] rgdal_1.1-1     maptools_0.8-37 sp_1.2-1
>>>
>>> loaded via a namespace (and not attached):
>>> [1] tools_3.2.2     foreign_0.8-66  grid_3.2.2      lattice_0.20-33
>>>
>>>
>>> Cheers, Mike.
>>>
>>>
>>>
>>>
>>>> I will attach images of one complex polygon, before and after saving
>>>> with writeOGR.
>>>>
>>>> Jim
>>>>
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>> 	[[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; fax +47 55 95 91 00
> e-mail: Roger.Bivand at nhh.no
>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: Roger.Bivand at nhh.no