[R-sig-Geo] rgdal: About the new gdal3 and proj >=6 condition

[Roger Bivand]

On Fri, 29 May 2020, Patrick Schratz wrote:

> The just released rgdal v1.5.8 requires gdal >= 3 when proj >= 6 is 
> present. It does not even install the package if this condition is not 
> met but errors during linking.

Please document with the output of 00install.out from R CMD check 
rgdal_1.5-8.tar.gz, and pkg-config proj --modversion, gdalinfo --version 
with system details.

Yes, PROJ >= 6 makes no sense without GDAL >= 3; GDAL 3 makes no sense 
with PROJ < 6. GDAL 3 with PROJ < 6 uses the old proj_api.h API and the 
old metadata structure with Proj4 string degradation; GDAL < 3 with PROJ 
>= 6 must use the deprecated API and the new metadata structure.

>
> I could not find any sources in the web or in the announcement post of
> v1.5.8 why this condition was enforced so strictly.
> Does it cause unwanted results?

Yes, definitely. GDAL 3 was released as 3, not 2.5, to stress the complete 
break between GDAL 2 (with PROJ <= 5) and GDAL >= 3 (with PROJ >= 6) after 
GDAL barnraising. So:

           PROJ
           < 6 >= 6 
GDAL < 3   OK   NO
     >= 3   NO   OK



> If so, then there is a big issue since the “gdal2 and proj >= 6”
> combination has been used by many people in the past already.

No good precedent, just binary packagers protecting slow downstream 
adaptation.

> I haven’t tracked down for how long this situation was on the market
> already.
> Also the {sf} package is still installable with this combination and I
> am not aware of any warnings/issues that came up due to this so far.
>

rgdal is a much older package and has much more older code that needs this 
protection. It is possible to accommodate the no-go areas, but I'd really 
value patches to R-forge, as I cannot check multiple PROJ/GDAL version 
combinations just because someone isn't up to speed.

> It further causes complete breakage for people relying on the homebrew
> package manager on macOS since current versions are gdal v2.2.4 and proj
> 7.0.1.

They should never have gone beyond PROJ 5 if they are stuck on GDAL. PROJ 
7 is a very different animal.

> The update to gdal3 is blocked since months due to an incompatibility
> with `liblas` (https://github.com/libLAS/libLAS/issues/164). Reading the
> issue, it looks unclear when this issue will be resolved.
> The alternative osgeo4mac tap holds formula that is unstable and
> somewhat broken. One cannot link rgdal with the current gdal v3.0.1 from
> osgeo4mac.
>
> This incompatibility with homebrew-core formula leads to further issues
> for CI tests that rely on the spatial homebrew stack for macOS builds.
>
> All of this is really unfortunate and I am wondering if this was a known
> factor when introducing this condition in the new release.
> Unfortunately rgdal is not hosted on any Git* provider (there is just
> the CRAN mirror with not much information about the recent changes) and
> it is unclear/untransparent to me if there is a continuous integration
> setup at all for this package.

Do not require me to leave the excellent SVN service on R-forge. Provide 
patches there.

> I hope this is not another case in the series “I do not like operating
> system XY and hence I do not test on it” which was seen recently in
> another package. That’s what CI is for and the responsibility of
> package authors.
>

I run and develop on Fedora, I  have no access to OSX, and the current 
release was fully checked with ~ 900 revdeps repeatedly since November on 
successive older and newer versions of PROJ and GDAL. I blocked new PROJ 
with old GDAL recently to simplify development. CI is only as good as the 
scripts, I am completely sure that repeated revdep testing and reading the 
output is superior.

> Also one should be aware that not every rgdal user is member of this
> mailing list (I guess not even 5%) and many people in production will
> face this error during install, most of them without any clue what to do
> or an idea why this was raised.
>

There has been plenty of information given about the CRS changes. rgdal 
could simply have been abandoned by me, and all  those production 
volunteers might have fixed things, but I never hear anything at all.

> In summary it would be great if
>
> - There would be more detailed information on the introduction of the
> new condition
> - Awareness of state-of-the-art platform related library versions before
> making such a release
> - Transparency/Existence of a cross-platform build process
> - If the incompatibility is just due to some features not being
> accessible with this configuration, I suggest to raise a warning during
> package load but not prevent the package from installing in the first
> place
> - A version-based NEWS file would be available. Currently one only sees
> a revision-based Changelog:
> https://cran.r-project.org/web/packages/rgdal/ChangeLog
>
> I am well aware that I am indirectly criticising (hopefully in a
> constructive way) the transparency and release process here, for which I
> will probably get a rough response.
> However, if that leads to more robustness and transparency in future
> release, I am fine with this.
>
> A quick patch release resolving the breakage would be highly
> appreciated.

Only with community imput. what you ask is not needed, just extra 
complexity. Please provide patches, or accept my invitation to join the 
R-forge project and commit your fixes directly. I can see how to do it, 
but I don't think it makes sense, and your messsage has not motivated me, 
to be honest. I'm prioritizing working with CRAN to iron out reverse 
dependency problems.

Roger

>
> Best, Patrick
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en