[R-sig-Geo] CRS in rgdal does not function with PROJ 6.2.0

[Cal Abel]

I have a Ubuntu 18.0.4 with R 3.6.1 system with the following releases from OSGeo for GEOS 3.8.0, GDAL 3.0.1, PROJ 6.2.0  I have an application that requires using gstat and thus I have to use rgdal as sf objects/classes aren’t supported in gstat, only sp. (This is a bit frustrating because rpostgis uses sf, so when I import my data it is an sf datatype, convert it to sp for gstat, and then back to sf to upload…) I thought the issue originally resided in sp, but after reviewing the sp source, it comes down to the calls that the sp CRS function makes using rgdal::. Below is a summary that shows my problem, sf links with the necessary system files and gets the correct CRS, rgdal does not. I think the issue is in how proj_api.h is included, specifically with not recognizing proj.db and instead relying on proj_def.dat which was removed in PROJ 6.1 as I recall. I can confirm that there is only one set of OSGeo libraries installed and those are those are the ones that I’ve compiled and are in /usr/local.

I can trick rgdal to use the proj_def.dat file by adding gdal/ and proj/ directories from the latest windows binary *.zip to ~/R/x86_64-pc-linux-gnu-library/3.6/rgdal. After doing this, rgdal will execute CRS, but complains that it can’t find proj.db   Once those files are removed it finds proj.db, but then can’t do CRS stuff. Win some, loose some I guess…

After reading through a bit of the source, it seems that rgdal defaults to using the deperecated proj4 strings and does not access the newer formats in proj.db even though during compilation rgdal finds the db with no problem. I am not too familiar with either package, so this is conjecture based on my limited testing/research.

As a note, if $PROJ_LIB is specified as an environment variable the result is Path to PROJ.4 shared files: /usr/local/share/proj.  But, when it is not specified the result is Path to PROJ.4 shared files: (autodetected). The behavior of rgdal remains unchanged in either case.


> library(sf)
Linking to GEOS 3.8.0, GDAL 3.0.1, PROJ 6.2.0
> library(rgdal)
Loading required package: sp
rgdal: version: 1.5-1, (SVN revision 879)
 Geospatial Data Abstraction Library extensions to R successfully loaded
 Loaded GDAL runtime: GDAL 3.0.1, released 2019/06/28
 Path to GDAL shared files:
 GDAL binary built with GEOS: TRUE
 Loaded PROJ.4 runtime: Rel. 6.2.0, September 1st, 2019, [PJ_VERSION: 620]
 Path to PROJ.4 shared files: /usr/local/share/proj
 Linking to sp version: 1.3-1
> CRS("+init=epsg:4269")
Error in CRS("+init=epsg:4269") : no arguments in initialization list
> st_crs(4269)
Coordinate Reference System:
  EPSG: 4269
  proj4string: "+proj=longlat +datum=NAD83 +no_defs”

Here is a coordinate transformation from NAD83 CRS to the UTM16 NAD83(2011) projection to show basic GDAL/PROJ6 functionality:
$ cs2cs EPSG:4269 EPSG:6345
35 -84
773798.10	3877156.69 0.00

I had forgotten that I originally reported the issue here https://github.com/edzer/sp/issues/59 <https://github.com/edzer/sp/issues/59> which I’ve since closed in testing for this email.

Below is an email exchange that drove this current email. I am unconvinced that Roger is correct. GDAL/PROJ work just fine, as does sf. A user error that he is referring to would cause those not to work/complain. The fact that I can make it work and make it not work should actually be grounds for successful bug identification.

Cal

> On Wed, 6 Nov 2019, Cal Abel wrote:
> 
> Dr. Bivand,
> 
> I attempted to report a bug in rgdal but don't really have any place to put it. I reported it in the OSGeo/gdal git page, but the issue was closed:
> 
> https://github.com/OSGeo/gdal/issues/1988 <https://github.com/OSGeo/gdal/issues/1988>
> I posted:
> 
> Almost certainly user blunder - neither **sf** nor **rgdal** install data/share files on source install, always relying on system installs. OP almost certainly neglected to read install notes. If GDAL and PROJ utilities can find their data/share files outside R, not finding them within R could signal R packages built against a different *.so than the one found at runtime.
> 
> 
> 
> One update to the above, I tried creating symbolic links to the compiled gdal and proj directories instead of copying the windows folders and it did not work. I also tried various combinations of links/files and was not able to get it to work. The only method that worked was to copy both windows directories.
> 
> The temporary fix I have, while it does not produce an error it now produces warnings:"
> 
> Do not try any fixes, you are almost certainly in error. How many versions of GDAL and PROJ are on your platform? For source installs, there should be one only, and GDAL should be built with that PROJ.
> 
> 
> Warning messages:
> 1: In CPL_crs_from_epsg(as.integer(x)) :
> GDAL Error 1: PROJ: proj_create_from_database: Cannot find proj.db
> 2: In CPL_crs_from_epsg(as.integer(x)) :
> GDAL Error 1: PROJ: proj_create_from_database: Cannot find proj.db
> 
> "
> 
> 
> Since it cannot find the PROJ proj.db, your installation of PROJ is broken, and copying across from an older version which does not have this key file will obviously fail.
> 
> In the future, what is the best avenue for reporting bugs?
> 
> 
> Use the R-sig-geo list. The package is not on github and will not be migrated - it is where https://cran.r-project.org/package=rgdal <https://cran.r-project.org/package=rgdal> says it is: https://r-forge.r-project.org/projects/rgdal/ <https://r-forge.r-project.org/projects/rgdal/> under SVN. All corresponndence through R-sig-geo, because others can correct your misapprehensions.
> 
> Note that the nabble link is totally different, which you should have known.
> 
> Further be aware that your usage pattern (+init=) is no longer valid in GDAL 3 and PROJ 6, and will stop working shortly. Your working example has GDAL 2 and PROJ 5, although this is not the only cause here, I think. It would be to your benefit to read https://proj.org/ <https://proj.org/> and https://gdal.org/ <https://gdal.org/>, and grasp https://gdal.org/development/rfc/rfc73_proj6_wkt2_srsbarn.html <https://gdal.org/development/rfc/rfc73_proj6_wkt2_srsbarn.html>, since this is a work-related query. How you migrate your corporate workflows to GDAL 3 and PROJ 6//7 may be of considerable interest (blog, mailing list post), and I'm spending all my time trying to adapt rgdal for this major change, and do not have patience with wasted time. See https://github.com/r-spatial/sf/issues/1146 <https://github.com/r-spatial/sf/issues/1146> and https://github.com/r-spatial/discuss/issues/28 <https://github.com/r-spatial/discuss/issues/28>.
> 
> Roger Bivand
> 
Cal Abel, PhD
crabel using signalpowerandlight.com <mailto:crabel using signalpowerandlight.com>

www.signalpowerandlight.com <http://www.signalpowerandlight.com/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20191108/8704eb5c/attachment.html>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 874 bytes
Desc: Message signed with OpenPGP
URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20191108/8704eb5c/attachment.sig>