[R-sig-Geo] rgdal_1.1-1: dsn path string problems

[Rainer Hurling]

Am 08.11.15 um 20:57 schrieb Roger Bivand:
> On Sun, 8 Nov 2015, Rainer Hurling wrote:
>
>> Hi Roger,
>>
>> many thanks for your quick answer.
>>
>>
>> Am 08.11.15 um 18:02 schrieb Roger Bivand:
>>> On Sun, 8 Nov 2015, Rainer Hurling wrote:
>>>
>>>> Dear Roger, dear list,
>>>>
>>>> many thanks for the newest update 1.1-1 of package rgdal. It is really
>>>> helpful.
>>>>
>>>> I am using rgdal with R-devel and GDAL 2.0.1, both on FreeBSD
>>>> 11.0-CURRENT and Windows7. On both platforms, I have problems with path
>>>> strings in dsn.
>>>>
>>>>
>>>>
>>>> (a) The minor problem is on Windows. While a dsn="C:/some/path" works
>>>> fine, the same path with a slash at the end (dsn="C:/some/path/")
>>>> fails.
>>>>
>>>> On most other software it is conventional to allow a closing slash.
>>>> Wouldn't it be nice, if rgdal would be more tolerant about it?
>>>>
>>>
>>> R base::file.exists does include this in its help file: "However,
>>> directory names must not include a trailing backslash or slash on
>>> Windows".
>>>
>>> so this isn't just rgdal. The dsn= argument for many drivers is a file
>>> name, not a directory; a work-around seems like overkill.
>>
>> OK, that seems reasonable. Of course, rgdal should handle this like R in
>> total. Thanks you for clarification.
>>
>>>>
>>>> Now my main problem:
>>>> (b) On FreeBSD, for some time now, the dsn path can not be used anymore
>>>> to check for existence of a file.
>>>>
>>>> I want to use writeOGR to overwrite an existing shapefile. I use
>>>> something like dsn="/path/to/somewhere" and layer="layername".
>>>>
>>>> Within writeOGR(), R/ogr_write.R calls the C++ function
>>>> 'ogrCheckExists'. ogrCheckExists is not able to return TRUE, if a
>>>> shapefile already exists. Every time its result is FALSE:
>>>>
>>>> R/ogr_write.R
>>>> 61:        ogrI <- .Call("ogrCheckExists", as.character(dsn),
>>>> s.character(layer), PACKAGE = "rgdal")
>>>> [1] FALSE
>>>>
>>>>
>>>> As far, as I can see, 'ogrCheckExists' comes from src/ogrsource.cpp.
>>>> Something must go wrong in that C++ file, at least for FreeBSD :(
>>>>
>>>> Unfortunalety, I have no skills how to debug such a C++ code within
>>>> an R
>>>> package ...
>>>>
>>>
>>> I do not have access to such a system. As a first step, does
>>> rgdal::ogrListLayers work correctly?
>>
>> The syntax, shown in the helpfile, does not work for me:
>>
>> rgdal::ogrListLayers(dsn="/path", layer="layername")
>> Fehler in rgdal::ogrListLayers(dsn = "/path", layer = "layername") :
>>  unbenutztes Argument (layer = "layername")    # unused argument
>>
>
> The helpfile usage section says:
>
> ogrListLayers(dsn)
>
> which works, as you see below, there is no layer argument, as the
> function queries the dsn to ask what layers it contains (differs between
> drivers).
>

Ok, as always, I should read documents more careful ;) Thanks for 
clarification again. It seems, the GDAL2 driver for ESRI shapefiles is a 
very special thing ...

>>
>> The same, with layer integrated in the dsn, works fine:
>>
>> rgdal::ogrListLayers(dsn="/path/layername.shp")
>> [1] "layername"
>> attr(,"driver")
>> [1] "ESRI Shapefile"
>> attr(,"nlayers")
>> [1] 1
>>
>>
>> This leads me to the idea, that the same could work for writeOGR or
>> ogrWrite.R. And yes, this works on FreeBSD:
>>
>> writeOGR(gis.layer, dsn="/path/layername.shp",
>>         layer="layername", driver="ESRI Shapefile",
>>         check_exists=TRUE, overwrite_layer=TRUE)
>>
>> So, I have to double 'layername' here, once in dsn and once in layer.
>> But it should be ok as a workaround until I found the real cause.
>>
>
> I have seen this before in GDAL2; it also happened in GDAL1 where the
> dsn directory contained for example *.dbf without *.shp and *.shx. The
> way dsn is handled does vary a lot between drivers.

I wonder, why it is necessary, to also use the layer="layername" (at 
least on FreeBSD with GDAL2), if the full path and filename in 
dsn="/path/layername.shp" is already given.

For now, I will use this dsn and layer notation for shapefiles with GDAL2.

>
>>>
>>> Beyond that, you'd need to insert Rprintf() statements into the C++ file
>>> to see whether the FALSE result comes from a failure to open the dsn
>>> with a known driver, or from the fact that such a layer is not found in
>>> that dsn.
>>
>> Thanks for the tip. I will investigate into it.
>>
>> A first question to this: If I want to integrate some Rprintf into
>> ogrsource.cpp in the section of ogrCheckExists (lines 832 ...), what is
>> a good syntax to output for example 'ogrSource' and 'Layer'?
>>
>> What I tried at ogrsource.cpp:l834 without success is:
>>
>> Rprintf( "ogrCheckExists: OGRSource %s %p\n", OGRLayer, (void *)
>> ogrSource) ;
>
> You'd need simple "No dns found" after line 860 and "No layer found"
> after line 878 to separate the two FALSE returns, but I guess the cause
> may have been in GDAL2, the shapefile driver, and/or what was in your
> dsn directory.

Ok, I was used to use print statements in debugging code also to get 
variables content, pointers and so on ...

Again, many many thanks for your quick and helpful comments. I now 
understand the driver dependend requirements of GDAL2 a lot more :)

Greetings from Göttingen,
Rainer

>
> Roger
>
>>
>> It think, I have to wrap 'ogrSource' with some function here? (I am not
>> a C/C++ programmer)
>>
>>>
>>> Does using a different driver help? Does using the shapefile.shp as the
>>> dsn help (I think this works better in GDAL2)?
>>
>> Because using the shapefile.shp as the dsn helps, I did not try a
>> different driver. If this is important for you, I can catch it up later.
>>
>> Thanks again for your help,
>> Rainer
>>
>>>
>>> Hope this helps,
>>>
>>> Roger
>>>
>>>> Any help is really appreciated. Please tell me, if I could try and test
>>>> something. My box has FreeBSD, R-devel, gcc-4.8.5, gdb-7.1.0 ...
>>>>
>>>> Greetings,
>>>> Rainer Hurling
>>>>
>>>>
>>>
>>
>>
>