[R-sig-Geo] rgdal: About the new gdal3 and proj >=6 condition
Roger Bivand
Roger@B|v@nd @end|ng |rom nhh@no
Wed Jun 3 17:30:09 CEST 2020
On Wed, 3 Jun 2020, Abdoulaye Sarr wrote:
> Have a problem with needed gdal version on mac
Please be much more specific. CRAN has macOS binaries of rgdal 1.5-8. Are
you trying to install all the external software dependencies then rgdal
from source? If this is difficult, rely on the CRAN macOS binaries, even
though the PROJ and GDAL versions do not yet support WKT2 CRS
representation. If you have installed PROJ and GDAL successfully, and PROJ
< 6 && GDAL < 3, or PROJ >= 6 && GDAL >= 3, you should be OK. If you have
the pointless PROJ >= 6 && GDAL < 3, you need the 1.5-9 tarball from
R-forge, download locally and run R CMD INSTALL
--configure-args="--with-proj_api=proj_api.h" rgdal_1.5-9.tar.gz.
Please report back on specifics, or simply use CRAN binaries and report
that.
We are actively looking for ways to prevent macOS users getting trapped
into trying to install from source when such a choice is not appropriate.
We have no control over the provenance of the installed software as we
have for Windows, thanks to Jeroen's rwinlib/gdal3 collection.
Roger
>
> On Tue, Jun 2, 2020 at 4:04 PM Robin Lovelace <rob00x using gmail.com> wrote:
>
>> Confirmed: this worked perfectly for me.
>>
>> Video evidence that this is fixed:
>> https://asciinema.org/a/zPQFbI6Q01oqUy1HRauCjLUXY
>>
>> Thanks Roger!
>>
>> Robin
>>
>> On Tue, Jun 2, 2020 at 4:00 PM Roger Bivand <Roger.Bivand using nhh.no> wrote:
>>
>>> Brian Ripley (thanks!) provided a patch for rgdal's configure.ac, and
>> the
>>> R-forge build and check processes now succeed, so please try to install
>>> either downloading
>>> http://download.r-forge.r-project.org/src/contrib/rgdal_1.5-9.tar.gz and
>>> installing from the command line (after running R CMD check), or
>>> install.packages("rgdal", repos="http://R-Forge.R-project.org").
>>>
>>> Remember that if GDAL is < 3, and PROJ >= 6, you need the configure
>>> argument: --configure-args=--with-proj_api=proj_api.h.
>>>
>>> Roger
>>>
>>> On Sun, 31 May 2020, Roger Bivand wrote:
>>>
>>>> Fresh version of rgdal rev 995 on link and R-forge, with more tweaks to
>>> the
>>>> configure file. Please test.
>>>>
>>>> Roger
>>>>
>>>> On Fri, 29 May 2020, Roger Bivand wrote:
>>>>
>>>>> I have put a tarball, built without vignettes with GDAL 2.2.4 and
>> PROJ
>>>>> 7.0.1 that passes CMD check (with warnings for missing vignettes) on:
>>>>>
>>>>> http://spatial.nhh.no/R/rgdal/gdal_1.5-9.tar.gz
>>>>>
>>>>> It involves code copying to provide duplicated functions in three
>>>>> settings:
>>>>>
>>>>> PROJ < 6 & GDAL < 3
>>>>> PROJ > = 6 & GDAL >= 3
>>>>> PROJ > = 6 & GDAL < 3 (your homebrew case)
>>>>>
>>>>> You need the configure argument:
>>>>> --configure-args=--with-proj_api=proj_api.h.
>>>>> Without the argument, you now get directed to use it.
>>>>>
>>>>> For now I've dropped the configure tests for sqlite3, curl and tiff;
>>> works
>>>>> for me but may not work for you.
>>>>>
>>>>> Please make the output of:
>>>>>
>>>>> R CMD check
>>> --install-args="--configure-args=--with-proj_api=proj_api.h"
>>>>> rgdal_1.5-9.tar.gz
>>>>>
>>>>> available ASAP. I count on an immediate response.
>>>>>
>>>>> Roger
>>>>>
>>>>> On Fri, 29 May 2020, Roger Bivand wrote:
>>>>>
>>>>>> On Fri, 29 May 2020, Patrick Schratz wrote:
>>>>>>
>>>>>>> The just released rgdal v1.5.8 requires gdal >= 3 when proj >= 6
>> is
>>>>>>> present. It does not even install the package if this condition
>> is
>>> not
>>>>>>> met
>>>>>>> but errors during linking.
>>>>>>
>>>>>> Please document with the output of 00install.out from R CMD check
>>>>>> rgdal_1.5-8.tar.gz, and pkg-config proj --modversion, gdalinfo
>>> --version
>>>>>> with system details.
>>>>>>
>>>>>> Yes, PROJ >= 6 makes no sense without GDAL >= 3; GDAL 3 makes no
>>> sense
>>>>>> with PROJ < 6. GDAL 3 with PROJ < 6 uses the old proj_api.h API and
>>> the
>>>>>> old metadata structure with Proj4 string degradation; GDAL < 3 with
>>> PROJ
>>>>>>> = 6 must use the deprecated API and the new metadata structure.
>>>>>>
>>>>>>>
>>>>>>> I could not find any sources in the web or in the announcement
>>> post of
>>>>>>> v1.5.8 why this condition was enforced so strictly.
>>>>>>> Does it cause unwanted results?
>>>>>>
>>>>>> Yes, definitely. GDAL 3 was released as 3, not 2.5, to stress the
>>>>>> complete
>>>>>> break between GDAL 2 (with PROJ <= 5) and GDAL >= 3 (with PROJ >=
>> 6)
>>>>>> after
>>>>>> GDAL barnraising. So:
>>>>>>
>>>>>> PROJ
>>>>>> < 6 >= 6
>>>>>> GDAL < 3 OK NO
>>>>>> > = 3 NO OK
>>>>>>
>>>>>>
>>>>>>
>>>>>>> If so, then there is a big issue since the “gdal2 and proj >= 6”
>>>>>>> combination has been used by many people in the past already.
>>>>>>
>>>>>> No good precedent, just binary packagers protecting slow downstream
>>>>>> adaptation.
>>>>>>
>>>>>>> I haven’t tracked down for how long this situation was on the
>>> market
>>>>>>> already.
>>>>>>> Also the {sf} package is still installable with this combination
>>> and I
>>>>>>> am not aware of any warnings/issues that came up due to this so
>>> far.
>>>>>>>
>>>>>>
>>>>>> rgdal is a much older package and has much more older code that
>> needs
>>>>>> this
>>>>>> protection. It is possible to accommodate the no-go areas, but I'd
>>>>>> really
>>>>>> value patches to R-forge, as I cannot check multiple PROJ/GDAL
>>> version
>>>>>> combinations just because someone isn't up to speed.
>>>>>>
>>>>>>> It further causes complete breakage for people relying on the
>>> homebrew
>>>>>>> package manager on macOS since current versions are gdal v2.2.4
>> and
>>>>>>> proj
>>>>>>> 7.0.1.
>>>>>>
>>>>>> They should never have gone beyond PROJ 5 if they are stuck on
>> GDAL.
>>>>>> PROJ
>>>>>> 7 is a very different animal.
>>>>>>
>>>>>>> The update to gdal3 is blocked since months due to an
>>> incompatibility
>>>>>>> with `liblas` (https://github.com/libLAS/libLAS/issues/164).
>>> Reading
>>>>>>> the
>>>>>>> issue, it looks unclear when this issue will be resolved.
>>>>>>> The alternative osgeo4mac tap holds formula that is unstable and
>>>>>>> somewhat broken. One cannot link rgdal with the current gdal
>> v3.0.1
>>>>>>> from
>>>>>>> osgeo4mac.
>>>>>>>
>>>>>>> This incompatibility with homebrew-core formula leads to further
>>>>>>> issues
>>>>>>> for CI tests that rely on the spatial homebrew stack for macOS
>>> builds.
>>>>>>>
>>>>>>> All of this is really unfortunate and I am wondering if this was
>> a
>>>>>>> known
>>>>>>> factor when introducing this condition in the new release.
>>>>>>> Unfortunately rgdal is not hosted on any Git* provider (there is
>>> just
>>>>>>> the CRAN mirror with not much information about the recent
>> changes)
>>>>>>> and
>>>>>>> it is unclear/untransparent to me if there is a continuous
>>> integration
>>>>>>> setup at all for this package.
>>>>>>
>>>>>> Do not require me to leave the excellent SVN service on R-forge.
>>> Provide
>>>>>> patches there.
>>>>>>
>>>>>>> I hope this is not another case in the series “I do not like
>>> operating
>>>>>>> system XY and hence I do not test on it” which was seen recently
>> in
>>>>>>> another package. That’s what CI is for and the responsibility of
>>>>>>> package authors.
>>>>>>>
>>>>>>
>>>>>> I run and develop on Fedora, I have no access to OSX, and the
>>> current
>>>>>> release was fully checked with ~ 900 revdeps repeatedly since
>>> November
>>>>>> on
>>>>>> successive older and newer versions of PROJ and GDAL. I blocked new
>>> PROJ
>>>>>> with old GDAL recently to simplify development. CI is only as good
>> as
>>>>>> the
>>>>>> scripts, I am completely sure that repeated revdep testing and
>>> reading
>>>>>> the
>>>>>> output is superior.
>>>>>>
>>>>>>> Also one should be aware that not every rgdal user is member of
>>> this
>>>>>>> mailing list (I guess not even 5%) and many people in production
>>> will
>>>>>>> face this error during install, most of them without any clue
>> what
>>> to
>>>>>>> do
>>>>>>> or an idea why this was raised.
>>>>>>>
>>>>>>
>>>>>> There has been plenty of information given about the CRS changes.
>>> rgdal
>>>>>> could simply have been abandoned by me, and all those production
>>>>>> volunteers might have fixed things, but I never hear anything at
>> all.
>>>>>>
>>>>>>> In summary it would be great if
>>>>>>>
>>>>>>> - There would be more detailed information on the introduction of
>>> the
>>>>>>> new condition
>>>>>>> - Awareness of state-of-the-art platform related library versions
>>>>>>> before
>>>>>>> making such a release
>>>>>>> - Transparency/Existence of a cross-platform build process
>>>>>>> - If the incompatibility is just due to some features not being
>>>>>>> accessible with this configuration, I suggest to raise a warning
>>>>>>> during
>>>>>>> package load but not prevent the package from installing in the
>>> first
>>>>>>> place
>>>>>>> - A version-based NEWS file would be available. Currently one
>> only
>>>>>>> sees
>>>>>>> a revision-based Changelog:
>>>>>>> https://cran.r-project.org/web/packages/rgdal/ChangeLog
>>>>>>>
>>>>>>> I am well aware that I am indirectly criticising (hopefully in a
>>>>>>> constructive way) the transparency and release process here, for
>>> which
>>>>>>> I
>>>>>>> will probably get a rough response.
>>>>>>> However, if that leads to more robustness and transparency in
>>> future
>>>>>>> release, I am fine with this.
>>>>>>>
>>>>>>> A quick patch release resolving the breakage would be highly
>>>>>>> appreciated.
>>>>>>
>>>>>> Only with community imput. what you ask is not needed, just extra
>>>>>> complexity. Please provide patches, or accept my invitation to join
>>> the
>>>>>> R-forge project and commit your fixes directly. I can see how to do
>>> it,
>>>>>> but I don't think it makes sense, and your messsage has not
>> motivated
>>>>>> me,
>>>>>> to be honest. I'm prioritizing working with CRAN to iron out
>> reverse
>>>>>> dependency problems.
>>>>>>
>>>>>> Roger
>>>>>>
>>>>>>>
>>>>>>> Best, Patrick
>>>>>>> [[alternative HTML version deleted]]
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> R-sig-Geo mailing list
>>>>>>> R-sig-Geo using r-project.org
>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, Norwegian School of Economics,
>>> Helleveien 30, N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
>>> https://orcid.org/0000-0003-2392-6140
>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
More information about the R-sig-Geo
mailing list