[R-sig-Geo] CRS in rgdal does not function with PROJ 6.2.0

Sat Nov 9 18:46:36 CET 2019

On Fri, 8 Nov 2019, Cal Abel wrote:

> I have a Ubuntu 18.0.4 with R 3.6.1 system with the following releases 
> from OSGeo for GEOS 3.8.0, GDAL 3.0.1, PROJ 6.2.0 I have an application 
> that requires using gstat and thus I have to use rgdal as sf 
> objects/classes aren’t supported in gstat, only sp. (This is a bit 
> frustrating because rpostgis uses sf, so when I import my data it is an 
> sf datatype, convert it to sp for gstat, and then back to sf to upload…)

First, note that gstat does support sf and stars class objects, as shown 
by https://keen-swartz-3146c4.netlify.com/interpolation.html (a draft 
chapter in a WIP-book).

Second, note that unless you are developing software, you should be using 
GDAL 2 and PROJ 5. You should not be using GDAL 3 and PROJ 6. For 
references, please do read the links I sent privately explaining why all 
open source spatial software using GDAL and PROJ is having difficulties 
adapting to the speed of chane in their APIs. Elsewhere I believe you 
encountered this with regard to libspatialite. Unless you really need to 
work on the wild side, stay on the safe side (GDAL2+PROJ5). sf with 
GDAL3/PROJ6 is also broken now.

You are working (day-time job) for a power company, and using R and gstat 
for free. You need to read more and write less before you impose on 
volunteer developers and maintainers whose day jobs do not allocate hours 
to handling your queries.

> I thought the issue originally resided in sp, but after reviewing the sp 
> source, it comes down to the calls that the sp CRS function makes using 
> rgdal::. Below is a summary that shows my problem, sf links with the 
> necessary system files and gets the correct CRS, rgdal does not. I think 
> the issue is in how proj_api.h is included, specifically with not 
> recognizing proj.db and instead relying on proj_def.dat which was 
> removed in PROJ 6.1 as I recall. I can confirm that there is only one 
> set of OSGeo libraries installed and those are those are the ones that 
> I’ve compiled and are in /usr/local.
>
> I can trick rgdal to use the proj_def.dat file by adding gdal/ and proj/ 
> directories from the latest windows binary *.zip to 
> ~/R/x86_64-pc-linux-gnu-library/3.6/rgdal. After doing this, rgdal will 
> execute CRS, but complains that it can’t find proj.db Once those files 
> are removed it finds proj.db, but then can’t do CRS stuff. Win some, 
> loose some I guess…
>

This was an issue faced years ago when PROJ was released without this file 
and is completely misleading here.

> After reading through a bit of the source, it seems that rgdal defaults 
> to using the deperecated proj4 strings and does not access the newer 
> formats in proj.db even though during compilation rgdal finds the db 
> with no problem. I am not too familiar with either package, so this is 
> conjecture based on my limited testing/research.
>
> As a note, if $PROJ_LIB is specified as an environment variable the 
> result is Path to PROJ.4 shared files: /usr/local/share/proj.  But, when 
> it is not specified the result is Path to PROJ.4 shared files: 
> (autodetected). The behavior of rgdal remains unchanged in either case.
>
>
>> library(sf)
> Linking to GEOS 3.8.0, GDAL 3.0.1, PROJ 6.2.0
>> library(rgdal)
> Loading required package: sp
> rgdal: version: 1.5-1, (SVN revision 879)
> Geospatial Data Abstraction Library extensions to R successfully loaded
> Loaded GDAL runtime: GDAL 3.0.1, released 2019/06/28
> Path to GDAL shared files:
> GDAL binary built with GEOS: TRUE
> Loaded PROJ.4 runtime: Rel. 6.2.0, September 1st, 2019, [PJ_VERSION: 620]
> Path to PROJ.4 shared files: /usr/local/share/proj
> Linking to sp version: 1.3-1
>> CRS("+init=epsg:4269")
> Error in CRS("+init=epsg:4269") : no arguments in initialization list
>> st_crs(4269)
> Coordinate Reference System:
>  EPSG: 4269
>  proj4string: "+proj=longlat +datum=NAD83 +no_defs”
>
> Here is a coordinate transformation from NAD83 CRS to the UTM16 NAD83(2011) projection to show basic GDAL/PROJ6 functionality:
> $ cs2cs EPSG:4269 EPSG:6345
> 35 -84
> 773798.10	3877156.69 0.00
>
> I had forgotten that I originally reported the issue here 
> https://github.com/edzer/sp/issues/59 
> <https://github.com/edzer/sp/issues/59> which I’ve since closed in 
> testing for this email.
>
> Below is an email exchange that drove this current email. I am 
> unconvinced that Roger is correct. GDAL/PROJ work just fine, as does sf. 
> A user error that he is referring to would cause those not to 
> work/complain. The fact that I can make it work and make it not work 
> should actually be grounds for successful bug identification.
>

You did not ask my permission to publish a private exchange on a list 
posting. It is normal practice to do so. Another time I might rather 
choose not to reply to any private queries, rather than offer advice, 
which had you heeded it, would have guided you in the right direction.

Check first, read a lot, pester others less, especially when your job 
depends on our offering  goodwill.

Reconfigure your system to use GDAL 2 and PROJ 5, the use whatever you 
need.

Roger

> Cal
>
>> On Wed, 6 Nov 2019, Cal Abel wrote:
>>
>> Dr. Bivand,
>>
>> I attempted to report a bug in rgdal but don't really have any place to 
>> put it. I reported it in the OSGeo/gdal git page, but the issue was 
>> closed:
>>
>> https://github.com/OSGeo/gdal/issues/1988 <https://github.com/OSGeo/gdal/issues/1988>
>> I posted:
>>
>> Almost certainly user blunder - neither **sf** nor **rgdal** install 
>> data/share files on source install, always relying on system installs. 
>> OP almost certainly neglected to read install notes. If GDAL and PROJ 
>> utilities can find their data/share files outside R, not finding them 
>> within R could signal R packages built against a different *.so than 
>> the one found at runtime.
>>
>>
>>
>> One update to the above, I tried creating symbolic links to the 
>> compiled gdal and proj directories instead of copying the windows 
>> folders and it did not work. I also tried various combinations of 
>> links/files and was not able to get it to work. The only method that 
>> worked was to copy both windows directories.
>>
>> The temporary fix I have, while it does not produce an error it now 
>> produces warnings:"
>>
>> Do not try any fixes, you are almost certainly in error. How many 
>> versions of GDAL and PROJ are on your platform? For source installs, 
>> there should be one only, and GDAL should be built with that PROJ.
>>
>>
>> Warning messages:
>> 1: In CPL_crs_from_epsg(as.integer(x)) :
>> GDAL Error 1: PROJ: proj_create_from_database: Cannot find proj.db
>> 2: In CPL_crs_from_epsg(as.integer(x)) :
>> GDAL Error 1: PROJ: proj_create_from_database: Cannot find proj.db
>>
>> "
>>
>>
>> Since it cannot find the PROJ proj.db, your installation of PROJ is 
>> broken, and copying across from an older version which does not have 
>> this key file will obviously fail.
>>
>> In the future, what is the best avenue for reporting bugs?
>>
>>
>> Use the R-sig-geo list. The package is not on github and will not be 
>> migrated - it is where https://cran.r-project.org/package=rgdal 
>> <https://cran.r-project.org/package=rgdal> says it is: 
>> https://r-forge.r-project.org/projects/rgdal/ 
>> <https://r-forge.r-project.org/projects/rgdal/> under SVN. All 
>> corresponndence through R-sig-geo, because others can correct your 
>> misapprehensions.
>>
>> Note that the nabble link is totally different, which you should have 
>> known.
>>
>> Further be aware that your usage pattern (+init=) is no longer valid in 
>> GDAL 3 and PROJ 6, and will stop working shortly. Your working example 
>> has GDAL 2 and PROJ 5, although this is not the only cause here, I 
>> think. It would be to your benefit to read https://proj.org/ 
>> <https://proj.org/> and https://gdal.org/ <https://gdal.org/>, and 
>> grasp https://gdal.org/development/rfc/rfc73_proj6_wkt2_srsbarn.html 
>> <https://gdal.org/development/rfc/rfc73_proj6_wkt2_srsbarn.html>, since 
>> this is a work-related query. How you migrate your corporate workflows 
>> to GDAL 3 and PROJ 6//7 may be of considerable interest (blog, mailing 
>> list post), and I'm spending all my time trying to adapt rgdal for this 
>> major change, and do not have patience with wasted time. See 
>> https://github.com/r-spatial/sf/issues/1146 
>> <https://github.com/r-spatial/sf/issues/1146> and 
>> https://github.com/r-spatial/discuss/issues/28 
>> <https://github.com/r-spatial/discuss/issues/28>.
>>
>> Roger Bivand
>>
> Cal Abel, PhD
> crabel using signalpowerandlight.com <mailto:crabel using signalpowerandlight.com>
>
> www.signalpowerandlight.com <http://www.signalpowerandlight.com/>
>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en