[R-sig-Geo] CRAN releases of sp, rgdal and rgeos

[Edzer Pebesma]

proj.5 contains new features, but none of these are, or will in the near
future be available to R users. All our (= Roger + me) efforts so far
have been to get software (rgdal, sf) running with proj.5, not to expose
any of its new features.
See also https://github.com/r-spatial/sf/issues/545

Also: if anyone insists on installing rgdal or sf from source on a Mac
but doesn't use the setup CRAN uses for building binaries, neither Roger
nor I can assist in case of problems beyond wildly guessing. Of course,
any help from anyone else on these kind of issues is highly appreciated.

On 06/12/2018 04:41 PM, Vijay Lulla wrote:
> Thank you for the explanation of difference between proj.h and proj_api.h
> and the recommendation of setting environment variable for pkg-config.  I
> had tried using pkg-config by setting env variables but haven't had much
> luck getting it to work.
> Cordially,
> Vijay.
> 
> On Tue, Jun 12, 2018 at 3:11 AM, Roger Bivand <Roger.Bivand using nhh.no> wrote:
> 
>> On Tue, 12 Jun 2018, Vijay Lulla wrote:
>>
>> I too use R on a mac but I cannot get rgdal to use proj5 (port info proj).
>>> rgdal still uses proj4 even when there is a newer version of proj already
>>> on my computer.  Neither of these proj packages (proj4 or proj) use
>>> pkg-config mechanism even though there are proj.pc files in both versions
>>> of the package.  Maybe using the pkg-config mechanism to detect version of
>>> packages during source installation might help?  I don't know whether this
>>> is a "proj" or a "macports" issue though.
>>>
>>>
>> Neither sp/rgdal nor sf plan to accomodate the proj.h interface until the
>> GDAL barn-raising project is completed. Even then, most software using proj
>> will have to support the proj_api.h and the proj.h APIs because many
>> platforms are bound to move at the speed of the slowest package using proj.
>> The proj_api.h API does use the new mechanisms internally, but it isn't
>> obvious how or whether proj.h pipelines should be exposed (GRASS has a
>> prototype implementation).
>>
>> If you work in geodetics and need sub-cm precision, this may matter; in
>> which case please follow the GDAL barn-raising project and provide use
>> cases where the proj.h API is required. The main difference is that the
>> proj_api.h API always transforms to the WGS84 datum first then transforms
>> to the target datum (many to one then one to many). When this was chosen
>> 1984 was not so long ago; the new proj.h API suggests transforming along a
>> pipeline from source to target datum without going through WGS84 (many to
>> many).
>>
>> If your question is about how to install rgdal from source on OSX, you may
>> need to ensure that you only have one version of proj installed, or use
>> configure arguments and load path modifications on your own responsibility.
>>
>> From rgdal 1.3-1, the pkg-config mechanism is used if the path to proj.pc
>> is set correctly (environment variable), but will fall back on standard
>> locations if pkg-config proj fails.
>>
>> Roger
>>
>>
>>
>>> On Mon, Jun 11, 2018 at 7:54 PM, Thiago V. dos Santos via R-sig-Geo <
>>> r-sig-geo using r-project.org> wrote:
>>>
>>> Hi Roger, thanks for the follow up. Answering to your comments:
>>>>
>>>> a) I am using Macports's gdal version 2.3.0.20180523. I am pretty sure
>>>> that its source code (i.e. the one that is compiled on my machine) is
>>>> downloaded from OSGEO (at least according to https://github.com/
>>>> macports/macports-ports/blob/master/gis/gdal/Portfile);
>>>>
>>>> b) I do have more than one libproj installed on my system (as QGIS for
>>>> macOS requires its own gdal and proj stuff), but only the Macports one is
>>>> on my path;
>>>>
>>>>
>>>> c) I tried running:
>>>> tools::testInstalledPackage("rgdal", outDir=tempdir())list.files(
>>>> tempdir())file.show(file.path(tempdir(), "rgdal-Ex.Rout"))
>>>>
>>>> but it unexpectedly returned "Error in library("rgdal") : there is no
>>>> package called ‘rgdal’. Execution halted".
>>>>
>>>> It looks weird but so far I am able to normally load and use at least a
>>>> couple of rgdal's functions.
>>>>
>>>> I will keep monitoring it though, and report any other issue.
>>>>
>>>> Cheers,
>>>>  -- Thiago V. dos Santos
>>>> Postdoctoral Research FellowDepartment of Climate and Space Science and
>>>> EngineeringUniversity of Michigan
>>>>
>>>>     On Sunday, June 10, 2018, 4:46:04 AM EDT, Roger Bivand <
>>>> Roger.Bivand using nhh.no> wrote:
>>>>
>>>>  Thanks for reporting, comments inline below (note that I have no OSX
>>>> access at all):
>>>>
>>>> On Sat, 9 Jun 2018, Thiago V. dos Santos wrote:
>>>>
>>>> Dear Roger,
>>>>> Thank you very much for the excellent work done with those packages.
>>>>> Today I update both rgeos and rgdal on my system - macOS 10.13.5 with
>>>>> all dependencies installed via MacPorts. I had updated sp a few days
>>>>> earlier. The dependencies versions on my system are gdal
>>>>> @2.3.0.20180523_0+grib+hdf4+hdf5+jasper+mpich+netcdf and proj @5.1.0_0.
>>>>> Rgeos's update went flawlessly, but I got a few errors while updating
>>>>> rgdal. It still compiled successfully, but I am concerned that some
>>>>> functionality might be compromised due to the errors.
>>>>> This is what I got (I have to manually specify the location of proj or
>>>>> rgdal won't find it):> install.packages('rgdal', type = "source",
>>>>> configure.args=c(
>>>>> +    '--with-proj-include=/opt/local/lib/proj5/include',
>>>>> +    '--with-proj-lib=/opt/local/lib/proj5/lib'))
>>>>> Installing package into ‘/Users/thiago/Documents/R-packages’
>>>>> (as ‘lib’ is unspecified)
>>>>> trying URL 'https://cran.r-project.org/src/contrib/rgdal_1.3-2.tar.gz'
>>>>> Content type 'application/x-gzip' length 1667049 bytes (1.6 MB)
>>>>> ==================================================
>>>>> downloaded 1.6 MB
>>>>>
>>>>> * installing *source* package ‘rgdal’ ...
>>>>> ** package ‘rgdal’ successfully unpacked and MD5 sums checked
>>>>> configure: CC: /usr/bin/clang
>>>>> configure: CXX: /usr/bin/clang++
>>>>> configure: rgdal: 1.3-2
>>>>> checking for /usr/bin/svnversion... yes
>>>>> configure: svn revision: 755
>>>>> checking whether the C++ compiler works... yes
>>>>> checking for C++ compiler default output file name... a.out
>>>>> checking for suffix of executables...
>>>>> checking whether we are cross compiling... no
>>>>> checking for suffix of object files... o
>>>>> checking whether we are using the GNU C++ compiler... yes
>>>>> checking whether /usr/bin/clang++ accepts -g... yes
>>>>> checking whether /usr/bin/clang++ supports C++11 features by default...
>>>>>
>>>> no
>>>>
>>>>> checking whether /usr/bin/clang++ supports C++11 features with
>>>>>
>>>> -std=gnu++11... yes
>>>>
>>>>> configure: C++11 support available
>>>>> checking for gdal-config... /opt/local/bin/gdal-config
>>>>> checking gdal-config usability... yes
>>>>> configure: GDAL: 2.4.0
>>>>>
>>>>
>>>> All OK up to the GDAL version returned by gdal-config - are you using the
>>>> released GDAL 2.3.0 (probably not) or master?
>>>>
>>>> checking C++11 support for GDAL >= 2.3.0... yes
>>>>> checking GDAL version >= 1.11.4... yes
>>>>> checking gdal: linking with --libs only... yes
>>>>> checking GDAL: /opt/local/share/gdal/pcs.csv readable... yes
>>>>> checking proj_api.h presence and usability... yes
>>>>> ./configure: line 3395: test: =: unary operator expected
>>>>>
>>>>
>>>> Will check, that line is:
>>>>
>>>> if test ${PROJ_VERSION} = "" ; then
>>>>
>>>> from configure.ac line 305. Possibly a shell dialect issue.
>>>>
>>>> checking PROJ version >= 4.8.0... yes
>>>>> checking projects.h presence and usability... yes
>>>>>
>>>>
>>>> These relate to configure.ac lines 376-419, and the outcome: epsg found
>>>> and readable is OK - could there be two libproj on your system (maybe for
>>>> different architectures)?
>>>>
>>>> Undefined symbols for architecture x86_64:
>>>>>  "_pj_ctx_fclose", referenced from:
>>>>>      _main in proj_conf_test2-06fe7d.o
>>>>>  "_pj_get_default_ctx", referenced from:
>>>>>      _main in proj_conf_test2-06fe7d.o
>>>>>  "_pj_open_lib", referenced from:
>>>>>      _main in proj_conf_test2-06fe7d.o
>>>>> ld: symbol(s) not found for architecture x86_64
>>>>> clang: error: linker command failed with exit code 1 (use -v to see
>>>>>
>>>> invocation)
>>>>
>>>>> ./configure: line 3511: ./proj_conf_test2: No such file or directory
>>>>> checking PROJ.4: epsg found and readable... yes
>>>>>
>>>>
>>>> Same here for next block in configure.ac; conus found and readable.
>>>>
>>>> Undefined symbols for architecture x86_64:
>>>>>  "_pj_ctx_fclose", referenced from:
>>>>>      _main in proj_conf_test3-3b7aa2.o
>>>>>  "_pj_get_default_ctx", referenced from:
>>>>>      _main in proj_conf_test3-3b7aa2.o
>>>>>  "_pj_open_lib", referenced from:
>>>>>      _main in proj_conf_test3-3b7aa2.o
>>>>> ld: symbol(s) not found for architecture x86_64
>>>>> clang: error: linker command failed with exit code 1 (use -v to see
>>>>>
>>>> invocation)
>>>>
>>>>> ./configure: line 3570: ./proj_conf_test3: No such file or directory
>>>>> checking PROJ.4: conus found and readable... yes
>>>>> configure: Package CPP flags:  -I/opt/local/include
>>>>>
>>>> -I/opt/local/lib/proj5/include
>>>>
>>>>> configure: Package LIBS:  -L/opt/local/lib -lgdal
>>>>>
>>>> -L/opt/local/lib/proj5/lib -lproj
>>>>
>>>>> configure: creating ./config.status
>>>>> config.status: creating src/Makevars
>>>>> ** libs
>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>> work/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2 -c OGR_write.cpp -o OGR_write.o
>>>>
>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>> work/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2 -c gdal-bindings.cpp -o gdal-bindings.o
>>>>
>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2  -c init.c -o init.o
>>>>
>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2  -c inverser.c -o inverser.o
>>>>
>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2  -c local_stubs.c -o local_stubs.o
>>>>
>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>> work/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2 -c ogr_geom.cpp -o ogr_geom.o
>>>>
>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2  -c ogr_polygons.c -o ogr_polygons.o
>>>>
>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>> work/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2 -c ogr_proj.cpp -o ogr_proj.o
>>>>
>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>> work/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2 -c ogrdrivers.cpp -o ogrdrivers.o
>>>>
>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>> work/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2 -c ogrsource.cpp -o ogrsource.o
>>>>
>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>> work/Resources/include"
>>>>>
>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>>>> -fPIC  -Wall -g -O2 -c projectit.cpp -o projectit.o
>>>>
>>>>> clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names
>>>>>
>>>> -undefined dynamic_lookup -single_module -multiply_defined suppress
>>>> -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o
>>>> rgdal.so OGR_write.o gdal-bindings.o init.o inverser.o local_stubs.o
>>>> ogr_geom.o ogr_polygons.o ogr_proj.o ogrdrivers.o ogrsource.o projectit.o
>>>> -L/opt/local/lib -lgdal -L/opt/local/lib/proj5/lib -lproj
>>>> -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework
>>>> -Wl,CoreFoundation
>>>>
>>>>> installing to /Users/thiago/Documents/R-packages/rgdal/libs
>>>>> ** R
>>>>> ** data
>>>>> ** inst
>>>>> ** byte-compile and prepare package for lazy loading
>>>>> ** help
>>>>> *** installing help indices
>>>>> ** building package indices
>>>>> ** installing vignettes
>>>>> ** testing if installed package can be loaded
>>>>> * DONE (rgdal)
>>>>>
>>>>> The downloaded source packages are in
>>>>>
>>>>> ‘/private/var/folders/_z/01gg71zs19g816v6m2dddt8w0000gn
>>>>>
>>>> /T/Rtmp5K6lt8/downloaded_packages’
>>>>
>>>>> ​Is this something that needs to be investigated, or I can just safely
>>>>> ignore those messages?
>>>>>
>>>>
>>>> Please try to run:
>>>>
>>>> tools::testInstalledPackage("rgdal", outDir=tempdir())
>>>> list.files(tempdir())
>>>> file.show(file.path(tempdir(), "rgdal-Ex.Rout"))
>>>>
>>>> and if the outcomes are as expected, you should be OK, please let us
>>>> know.
>>>>
>>>> If any OSX users installing from source can contribute, that would be
>>>> useful. It will take a little while before the CRAN OSX binaries catch up
>>>> with new PROJ and GDAL.
>>>>
>>>> Again thanks for reporting!
>>>>
>>>> Roger
>>>>
>>>> Many thanks, -- Thiago V. dos Santos
>>>>> Postdoctoral Research FellowDepartment of Climate and Space Science and
>>>>>
>>>> EngineeringUniversity of Michigan
>>>>
>>>>>
>>>>>    On Friday, June 8, 2018, 2:15:43 PM EDT, Roger Bivand <
>>>>>
>>>> Roger.Bivand using nhh.no> wrote:
>>>>
>>>>>
>>>>> There are new releases of sp, rgdal and rgeos on CRAN. Please install sp
>>>>> first, then the other two, which link to the installed sp. They all
>>>>> address so-called rchk issues, which have not so far been a problem, but
>>>>> might have become more fragile as R's internal memory management is made
>>>>> even more efficient. This involves compiled code using memory allocated
>>>>>
>>>> by
>>>>
>>>>> R to be freed by R's garbage collector, which has to know if an object
>>>>> is
>>>>> still being used. Tomas Kalibera, the author of rchk, helped resolve and
>>>>> explain the issues encountered - what was good coding practice fifteen
>>>>> years ago isn't always still good practice.
>>>>>
>>>>> In addition, the earliest versions of GDAL and PROJ with which rgdal
>>>>> will
>>>>> work have been updated, and set to PROJ 4.8.0 and GDAL 1.11.4. The
>>>>>
>>>> current
>>>>
>>>>> released versions of PROJ and GDAL are to be prefered, as bugs have been
>>>>> fixed and new features and drivers introduced. A check has been put
>>>>> in place to trap attempts to install rgdal without a C++11-capable
>>>>> compiler when the GDAL version is >=2.3.0 - which requires C++11. rgeos
>>>>>
>>>> is
>>>>
>>>>> ready for the forthcoming version of GEOS.
>>>>>
>>>>> The CRAN team has also been very supportive of our efforts to bring
>>>>> compiled code in these packages into rchk compliance.
>>>>>
>>>>> Please get in touch if you see any loose ends in these releases.
>>>>>
>>>>> Roger
>>>>>
>>>>>
>>>>>
>>>> --
>>>> Roger Bivand
>>>> Department of Economics, Norwegian School of Economics,
>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>> voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
>>>> http://orcid.org/0000-0003-2392-6140
>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>         [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo using r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>
>>>>
>>>
>>>
>>>
>>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
>> http://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>
> 
> 
> 

-- 
Edzer Pebesma
Institute for Geoinformatics
Heisenbergstrasse 2, 48151 Muenster, Germany
Phone: +49 251 8333081