[R-sig-Geo] rgdal_1.1-1: dsn path string problems

[Roger Bivand]

On Sun, 8 Nov 2015, Rainer Hurling wrote:

> Hi Roger,
>
> many thanks for your quick answer.
>
>
> Am 08.11.15 um 18:02 schrieb Roger Bivand:
>> On Sun, 8 Nov 2015, Rainer Hurling wrote:
>>
>>> Dear Roger, dear list,
>>>
>>> many thanks for the newest update 1.1-1 of package rgdal. It is really
>>> helpful.
>>>
>>> I am using rgdal with R-devel and GDAL 2.0.1, both on FreeBSD
>>> 11.0-CURRENT and Windows7. On both platforms, I have problems with path
>>> strings in dsn.
>>>
>>>
>>>
>>> (a) The minor problem is on Windows. While a dsn="C:/some/path" works
>>> fine, the same path with a slash at the end (dsn="C:/some/path/") fails.
>>>
>>> On most other software it is conventional to allow a closing slash.
>>> Wouldn't it be nice, if rgdal would be more tolerant about it?
>>>
>>
>> R base::file.exists does include this in its help file: "However,
>> directory names must not include a trailing backslash or slash on Windows".
>>
>> so this isn't just rgdal. The dsn= argument for many drivers is a file
>> name, not a directory; a work-around seems like overkill.
>
> OK, that seems reasonable. Of course, rgdal should handle this like R in
> total. Thanks you for clarification.
>
>>>
>>> Now my main problem:
>>> (b) On FreeBSD, for some time now, the dsn path can not be used anymore
>>> to check for existence of a file.
>>>
>>> I want to use writeOGR to overwrite an existing shapefile. I use
>>> something like dsn="/path/to/somewhere" and layer="layername".
>>>
>>> Within writeOGR(), R/ogr_write.R calls the C++ function
>>> 'ogrCheckExists'. ogrCheckExists is not able to return TRUE, if a
>>> shapefile already exists. Every time its result is FALSE:
>>>
>>> R/ogr_write.R
>>> 61:        ogrI <- .Call("ogrCheckExists", as.character(dsn),
>>> s.character(layer), PACKAGE = "rgdal")
>>> [1] FALSE
>>>
>>>
>>> As far, as I can see, 'ogrCheckExists' comes from src/ogrsource.cpp.
>>> Something must go wrong in that C++ file, at least for FreeBSD :(
>>>
>>> Unfortunalety, I have no skills how to debug such a C++ code within an R
>>> package ...
>>>
>>
>> I do not have access to such a system. As a first step, does
>> rgdal::ogrListLayers work correctly?
>
> The syntax, shown in the helpfile, does not work for me:
>
> rgdal::ogrListLayers(dsn="/path", layer="layername")
> Fehler in rgdal::ogrListLayers(dsn = "/path", layer = "layername") :
>  unbenutztes Argument (layer = "layername")    # unused argument
>

The helpfile usage section says:

ogrListLayers(dsn)

which works, as you see below, there is no layer argument, as the 
function queries the dsn to ask what layers it contains (differs between 
drivers).

>
> The same, with layer integrated in the dsn, works fine:
>
> rgdal::ogrListLayers(dsn="/path/layername.shp")
> [1] "layername"
> attr(,"driver")
> [1] "ESRI Shapefile"
> attr(,"nlayers")
> [1] 1
>
>
> This leads me to the idea, that the same could work for writeOGR or
> ogrWrite.R. And yes, this works on FreeBSD:
>
> writeOGR(gis.layer, dsn="/path/layername.shp",
>         layer="layername", driver="ESRI Shapefile",
>         check_exists=TRUE, overwrite_layer=TRUE)
>
> So, I have to double 'layername' here, once in dsn and once in layer.
> But it should be ok as a workaround until I found the real cause.
>

I have seen this before in GDAL2; it also happened in GDAL1 where the dsn 
directory contained for example *.dbf without *.shp and *.shx. The way dsn 
is handled does vary a lot between drivers.

>>
>> Beyond that, you'd need to insert Rprintf() statements into the C++ file
>> to see whether the FALSE result comes from a failure to open the dsn
>> with a known driver, or from the fact that such a layer is not found in
>> that dsn.
>
> Thanks for the tip. I will investigate into it.
>
> A first question to this: If I want to integrate some Rprintf into
> ogrsource.cpp in the section of ogrCheckExists (lines 832 ...), what is
> a good syntax to output for example 'ogrSource' and 'Layer'?
>
> What I tried at ogrsource.cpp:l834 without success is:
>
> Rprintf( "ogrCheckExists: OGRSource %s %p\n", OGRLayer, (void *)
> ogrSource) ;

You'd need simple "No dns found" after line 860 and "No layer found" after 
line 878 to separate the two FALSE returns, but I guess the cause may have 
been in GDAL2, the shapefile driver, and/or what was in your dsn 
directory.

Roger

>
> It think, I have to wrap 'ogrSource' with some function here? (I am not
> a C/C++ programmer)
>
>>
>> Does using a different driver help? Does using the shapefile.shp as the
>> dsn help (I think this works better in GDAL2)?
>
> Because using the shapefile.shp as the dsn helps, I did not try a
> different driver. If this is important for you, I can catch it up later.
>
> Thanks again for your help,
> Rainer
>
>>
>> Hope this helps,
>>
>> Roger
>>
>>> Any help is really appreciated. Please tell me, if I could try and test
>>> something. My box has FreeBSD, R-devel, gcc-4.8.5, gdb-7.1.0 ...
>>>
>>> Greetings,
>>> Rainer Hurling
>>>
>>>
>>
>
>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: Roger.Bivand at nhh.no