[R-sig-Geo] rgdal 1.1-1 bug?

Mon Dec 7 17:40:53 CET 2015

Thank you Tom and Roger for your attention to this issue.

Here is a link to a zipfile (17 MB).  It contains two folders/shapefiles.  One is the single big polygon with many holes that I had sent to Tom.  The other is the shapefile it was extracted from, with many additional polygons showing similar problems.  

https://www.dropbox.com/s/xkp8sdoccg6jyg4/offending_shapefile.zip?dl=0

These shapefiles are from Arc and have never been written by writeOGR, so should be fine when first opened.

For now, I have been unable to revert to the earlier version of rgdal as R complains that it doesn't match the R version.
Jim

-----Original Message-----
From: R-sig-Geo [mailto:r-sig-geo-bounces at r-project.org] On Behalf Of Roger Bivand
Sent: Saturday, December 05, 2015 3:50 AM
To: Tom Philippi
Cc: r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] rgdal 1.1-1 bug?

On Sat, 5 Dec 2015, Tom Philippi wrote:

> James & all--

If you make the offending shapefile available, I'll look at this in a week or so (no time before then). The possible change is from:

https://stat.ethz.ch/pipermail/r-sig-geo/2015-November/023656.html

and affected writing - before Shapefiles were written as-is, from 1.1-1 all output geometries without comments are put through
OGRGeometryFactory::organizePolygons() in GDAL/OGR. This may be a case in which the OGR function fails to organise the polygons in the intended fashion. I have a build of rgdal 1.1-2 for Windows using GDAL
2.0.1 (thanks again to Brian Ripley) which you may try, but it will most likely not change anything. Also, we don't know whether the input data (Polygons objects) have comments or not, but if read through readOGR(), the Shapefile driver also runs OGRGeometryFactory::organizePolygons() on read.

I need access to the offending shapefile to see whether alternative settings to OGRGeometryFactory::organizePolygons(), or a user option to avoid this very sensible function with possibly bad data, might alleviate the problem. Please put on URL or attach offlist.

Roger

> With your shapefile I can reproduce your problem under Windows 7x64 in 
> R
> 3.2.2 rgdal_1.1-1 sp_1.2-1, but not R 3.1.2 rgdal_0.9-3 sp_1.1-1 or 
> earlier.  Both rgdals are linked to gdal 1.11.2 released 2015/02/10.
> I also can reproduce the issue writing to & reading from GeoJSON 
> instead of shapefiles.  Files written by rgdal 1.1-1 have the lines 
> connecting holes when read by rgdal_0.9-3, but files written by 
> rgdal_0.9-3 are read fine by 1.1-1.
>
> The rarity about the shapefile in question is that it has 57 holes in 
> the same polygon, which (along with the lines connecting holes) 
> suggests that the issue might be related to SpatialPolygons not 
> assigning holes to specific (non-hole) polygons, see:
> http://finzi.psych.upenn.edu/R/library/rgeos/html/comment-functions.ht
> ml
>
> I don't have time (or ability) to go much further right now in terms 
> of finding the change, but perhaps someone else can?
>
> Tom 2
>
> On Fri, Dec 4, 2015 at 2:34 PM, Michael Sumner <mdsumner at gmail.com> wrote:
>
>>
>>
>> On Sat, 5 Dec 2015 at 05:26 Worrall, James -FS <jworrall at fs.fed.us> wrote:
>>
>>> [image: image001.gif]
>>>
>>> We have been happily using rgdal to read and write ESRI Shapefiles.
>>> Recently my colleague began using it.  We noticed that after saving 
>>> and reopening files, many polygons had been reconfigured, as if the 
>>> order of the vertices had become jumbled and there were many 
>>> straight lines cutting across the polygon.  I’m not sure if an image 
>>> can be pasted in this list but I’ll try.
>>>
>>> I did the same thing on my computer and the problem did not occur.  
>>> We found that I had an older version of rgdal, 1.0-7.  When I 
>>> updated it to 1.1-1, the problem occurred just as it had with her.
>>>
>>> Simply opening and saving a shapefile causes the problem and 
>>> essentially jumbles the shapefile.  The bug, if it is one, was 
>>> introduced after 1.0-7
>>>
>>
>>
>>
>>
>> I can't reproduce this. Can you provide a reproducible example?  
>> Also, "opening and saving a shapefiles" is not the way to describe 
>> what is done in the usual rgdal workflow, so it's not clear what you 
>> did. Here's what I
>> tried:
>>
>> library(maptools)
>> data(wrld_simpl)
>>
>> library(rgdal)
>> #rgdal: version: 1.1-1, (SVN revision 572) # Geospatial Data 
>> Abstraction Library extensions to R successfully loaded # Loaded GDAL 
>> runtime: GDAL 1.11.2, released 2015/02/10 # Path to GDAL shared 
>> files: E:/inst/R/R/library/rgdal/gdal # GDAL does not use iconv for 
>> recoding strings.
>> # Loaded PROJ.4 runtime: Rel. 4.9.1, 04 March 2015, [PJ_VERSION: 491] 
>> # Path to PROJ.4 shared files: E:/inst/R/R/library/rgdal/proj # 
>> Linking to sp version: 1.2-1
>>
>>
>> writeOGR(wrld_simpl, ".", "1.1-1", "ESRI Shapefile")
>> x1 <- readOGR(".", "1.1-1")
>> plot(x1)  ## all good
>>
>>
>>  sessionInfo()
>> R version 3.2.2 Patched (2015-08-27 r69201)
>> Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 8.1 
>> x64 (build 9600)
>>
>> locale:
>> [1] LC_COLLATE=English_Australia.1252  
>> LC_CTYPE=English_Australia.1252 [3] 
>> LC_MONETARY=English_Australia.1252 LC_NUMERIC=C [5] 
>> LC_TIME=English_Australia.1252
>>
>> attached base packages:
>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>
>> other attached packages:
>> [1] rgdal_1.1-1     maptools_0.8-37 sp_1.2-1
>>
>> loaded via a namespace (and not attached):
>> [1] tools_3.2.2     foreign_0.8-66  grid_3.2.2      lattice_0.20-33
>>
>>
>> Cheers, Mike.
>>
>>
>>
>>
>>> I will attach images of one complex polygon, before and after saving 
>>> with writeOGR.
>>>
>>> Jim
>>>
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo

--
Roger Bivand
Department of Economics, Norwegian School of Economics, Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: Roger.Bivand at nhh.no