[R-sig-Geo] CRAN releases of sp, rgdal and rgeos

[Roger Bivand]

On Tue, 12 Jun 2018, Vijay Lulla wrote:

> I too use R on a mac but I cannot get rgdal to use proj5 (port info proj).
> rgdal still uses proj4 even when there is a newer version of proj already
> on my computer.  Neither of these proj packages (proj4 or proj) use
> pkg-config mechanism even though there are proj.pc files in both versions
> of the package.  Maybe using the pkg-config mechanism to detect version of
> packages during source installation might help?  I don't know whether this
> is a "proj" or a "macports" issue though.
>

Neither sp/rgdal nor sf plan to accomodate the proj.h interface until the 
GDAL barn-raising project is completed. Even then, most software using 
proj will have to support the proj_api.h and the proj.h APIs because many 
platforms are bound to move at the speed of the slowest package using 
proj. The proj_api.h API does use the new mechanisms internally, but it 
isn't obvious how or whether proj.h pipelines should be exposed (GRASS has 
a prototype implementation).

If you work in geodetics and need sub-cm precision, this may matter; in 
which case please follow the GDAL barn-raising project and provide use 
cases where the proj.h API is required. The main difference is that the 
proj_api.h API always transforms to the WGS84 datum first then transforms 
to the target datum (many to one then one to many). When this was chosen 
1984 was not so long ago; the new proj.h API suggests transforming along a 
pipeline from source to target datum without going through WGS84 (many to 
many).

If your question is about how to install rgdal from source on OSX, you may 
need to ensure that you only have one version of proj installed, or use 
configure arguments and load path modifications on your own 
responsibility.

From rgdal 1.3-1, the pkg-config mechanism is used if the path to proj.pc 
is set correctly (environment variable), but will fall back on standard 
locations if pkg-config proj fails.

Roger

>
> On Mon, Jun 11, 2018 at 7:54 PM, Thiago V. dos Santos via R-sig-Geo <
> r-sig-geo using r-project.org> wrote:
>
>> Hi Roger, thanks for the follow up. Answering to your comments:
>>
>> a) I am using Macports's gdal version 2.3.0.20180523. I am pretty sure
>> that its source code (i.e. the one that is compiled on my machine) is
>> downloaded from OSGEO (at least according to https://github.com/
>> macports/macports-ports/blob/master/gis/gdal/Portfile);
>>
>> b) I do have more than one libproj installed on my system (as QGIS for
>> macOS requires its own gdal and proj stuff), but only the Macports one is
>> on my path;
>>
>>
>> c) I tried running:
>> tools::testInstalledPackage("rgdal", outDir=tempdir())list.files(
>> tempdir())file.show(file.path(tempdir(), "rgdal-Ex.Rout"))
>>
>> but it unexpectedly returned "Error in library("rgdal") : there is no
>> package called ‘rgdal’. Execution halted".
>>
>> It looks weird but so far I am able to normally load and use at least a
>> couple of rgdal's functions.
>>
>> I will keep monitoring it though, and report any other issue.
>>
>> Cheers,
>>  -- Thiago V. dos Santos
>> Postdoctoral Research FellowDepartment of Climate and Space Science and
>> EngineeringUniversity of Michigan
>>
>>     On Sunday, June 10, 2018, 4:46:04 AM EDT, Roger Bivand <
>> Roger.Bivand using nhh.no> wrote:
>>
>>  Thanks for reporting, comments inline below (note that I have no OSX
>> access at all):
>>
>> On Sat, 9 Jun 2018, Thiago V. dos Santos wrote:
>>
>>> Dear Roger,
>>> Thank you very much for the excellent work done with those packages.
>>> Today I update both rgeos and rgdal on my system - macOS 10.13.5 with
>>> all dependencies installed via MacPorts. I had updated sp a few days
>>> earlier. The dependencies versions on my system are gdal
>>> @2.3.0.20180523_0+grib+hdf4+hdf5+jasper+mpich+netcdf and proj @5.1.0_0.
>>> Rgeos's update went flawlessly, but I got a few errors while updating
>>> rgdal. It still compiled successfully, but I am concerned that some
>>> functionality might be compromised due to the errors.
>>> This is what I got (I have to manually specify the location of proj or
>>> rgdal won't find it):> install.packages('rgdal', type = "source",
>>> configure.args=c(
>>> +    '--with-proj-include=/opt/local/lib/proj5/include',
>>> +    '--with-proj-lib=/opt/local/lib/proj5/lib'))
>>> Installing package into ‘/Users/thiago/Documents/R-packages’
>>> (as ‘lib’ is unspecified)
>>> trying URL 'https://cran.r-project.org/src/contrib/rgdal_1.3-2.tar.gz'
>>> Content type 'application/x-gzip' length 1667049 bytes (1.6 MB)
>>> ==================================================
>>> downloaded 1.6 MB
>>>
>>> * installing *source* package ‘rgdal’ ...
>>> ** package ‘rgdal’ successfully unpacked and MD5 sums checked
>>> configure: CC: /usr/bin/clang
>>> configure: CXX: /usr/bin/clang++
>>> configure: rgdal: 1.3-2
>>> checking for /usr/bin/svnversion... yes
>>> configure: svn revision: 755
>>> checking whether the C++ compiler works... yes
>>> checking for C++ compiler default output file name... a.out
>>> checking for suffix of executables...
>>> checking whether we are cross compiling... no
>>> checking for suffix of object files... o
>>> checking whether we are using the GNU C++ compiler... yes
>>> checking whether /usr/bin/clang++ accepts -g... yes
>>> checking whether /usr/bin/clang++ supports C++11 features by default...
>> no
>>> checking whether /usr/bin/clang++ supports C++11 features with
>> -std=gnu++11... yes
>>> configure: C++11 support available
>>> checking for gdal-config... /opt/local/bin/gdal-config
>>> checking gdal-config usability... yes
>>> configure: GDAL: 2.4.0
>>
>> All OK up to the GDAL version returned by gdal-config - are you using the
>> released GDAL 2.3.0 (probably not) or master?
>>
>>> checking C++11 support for GDAL >= 2.3.0... yes
>>> checking GDAL version >= 1.11.4... yes
>>> checking gdal: linking with --libs only... yes
>>> checking GDAL: /opt/local/share/gdal/pcs.csv readable... yes
>>> checking proj_api.h presence and usability... yes
>>> ./configure: line 3395: test: =: unary operator expected
>>
>> Will check, that line is:
>>
>> if test ${PROJ_VERSION} = "" ; then
>>
>> from configure.ac line 305. Possibly a shell dialect issue.
>>
>>> checking PROJ version >= 4.8.0... yes
>>> checking projects.h presence and usability... yes
>>
>> These relate to configure.ac lines 376-419, and the outcome: epsg found
>> and readable is OK - could there be two libproj on your system (maybe for
>> different architectures)?
>>
>>> Undefined symbols for architecture x86_64:
>>>  "_pj_ctx_fclose", referenced from:
>>>      _main in proj_conf_test2-06fe7d.o
>>>  "_pj_get_default_ctx", referenced from:
>>>      _main in proj_conf_test2-06fe7d.o
>>>  "_pj_open_lib", referenced from:
>>>      _main in proj_conf_test2-06fe7d.o
>>> ld: symbol(s) not found for architecture x86_64
>>> clang: error: linker command failed with exit code 1 (use -v to see
>> invocation)
>>> ./configure: line 3511: ./proj_conf_test2: No such file or directory
>>> checking PROJ.4: epsg found and readable... yes
>>
>> Same here for next block in configure.ac; conus found and readable.
>>
>>> Undefined symbols for architecture x86_64:
>>>  "_pj_ctx_fclose", referenced from:
>>>      _main in proj_conf_test3-3b7aa2.o
>>>  "_pj_get_default_ctx", referenced from:
>>>      _main in proj_conf_test3-3b7aa2.o
>>>  "_pj_open_lib", referenced from:
>>>      _main in proj_conf_test3-3b7aa2.o
>>> ld: symbol(s) not found for architecture x86_64
>>> clang: error: linker command failed with exit code 1 (use -v to see
>> invocation)
>>> ./configure: line 3570: ./proj_conf_test3: No such file or directory
>>> checking PROJ.4: conus found and readable... yes
>>> configure: Package CPP flags:  -I/opt/local/include
>> -I/opt/local/lib/proj5/include
>>> configure: Package LIBS:  -L/opt/local/lib -lgdal
>> -L/opt/local/lib/proj5/lib -lproj
>>> configure: creating ./config.status
>>> config.status: creating src/Makevars
>>> ** libs
>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2 -c OGR_write.cpp -o OGR_write.o
>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2 -c gdal-bindings.cpp -o gdal-bindings.o
>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2  -c init.c -o init.o
>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2  -c inverser.c -o inverser.o
>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2  -c local_stubs.c -o local_stubs.o
>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2 -c ogr_geom.cpp -o ogr_geom.o
>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2  -c ogr_polygons.c -o ogr_polygons.o
>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2 -c ogr_proj.cpp -o ogr_proj.o
>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2 -c ogrdrivers.cpp -o ogrdrivers.o
>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2 -c ogrsource.cpp -o ogrsource.o
>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include"
>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>> -I"/Users/thiago/Documents/R-packages/sp/include" -I/usr/local/include
>> -fPIC  -Wall -g -O2 -c projectit.cpp -o projectit.o
>>> clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names
>> -undefined dynamic_lookup -single_module -multiply_defined suppress
>> -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o
>> rgdal.so OGR_write.o gdal-bindings.o init.o inverser.o local_stubs.o
>> ogr_geom.o ogr_polygons.o ogr_proj.o ogrdrivers.o ogrsource.o projectit.o
>> -L/opt/local/lib -lgdal -L/opt/local/lib/proj5/lib -lproj
>> -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework
>> -Wl,CoreFoundation
>>> installing to /Users/thiago/Documents/R-packages/rgdal/libs
>>> ** R
>>> ** data
>>> ** inst
>>> ** byte-compile and prepare package for lazy loading
>>> ** help
>>> *** installing help indices
>>> ** building package indices
>>> ** installing vignettes
>>> ** testing if installed package can be loaded
>>> * DONE (rgdal)
>>>
>>> The downloaded source packages are in
>>>
>>> ‘/private/var/folders/_z/01gg71zs19g816v6m2dddt8w0000gn
>> /T/Rtmp5K6lt8/downloaded_packages’
>>> ​Is this something that needs to be investigated, or I can just safely
>>> ignore those messages?
>>
>> Please try to run:
>>
>> tools::testInstalledPackage("rgdal", outDir=tempdir())
>> list.files(tempdir())
>> file.show(file.path(tempdir(), "rgdal-Ex.Rout"))
>>
>> and if the outcomes are as expected, you should be OK, please let us know.
>>
>> If any OSX users installing from source can contribute, that would be
>> useful. It will take a little while before the CRAN OSX binaries catch up
>> with new PROJ and GDAL.
>>
>> Again thanks for reporting!
>>
>> Roger
>>
>>> Many thanks, -- Thiago V. dos Santos
>>> Postdoctoral Research FellowDepartment of Climate and Space Science and
>> EngineeringUniversity of Michigan
>>>
>>>    On Friday, June 8, 2018, 2:15:43 PM EDT, Roger Bivand <
>> Roger.Bivand using nhh.no> wrote:
>>>
>>> There are new releases of sp, rgdal and rgeos on CRAN. Please install sp
>>> first, then the other two, which link to the installed sp. They all
>>> address so-called rchk issues, which have not so far been a problem, but
>>> might have become more fragile as R's internal memory management is made
>>> even more efficient. This involves compiled code using memory allocated
>> by
>>> R to be freed by R's garbage collector, which has to know if an object is
>>> still being used. Tomas Kalibera, the author of rchk, helped resolve and
>>> explain the issues encountered - what was good coding practice fifteen
>>> years ago isn't always still good practice.
>>>
>>> In addition, the earliest versions of GDAL and PROJ with which rgdal will
>>> work have been updated, and set to PROJ 4.8.0 and GDAL 1.11.4. The
>> current
>>> released versions of PROJ and GDAL are to be prefered, as bugs have been
>>> fixed and new features and drivers introduced. A check has been put
>>> in place to trap attempts to install rgdal without a C++11-capable
>>> compiler when the GDAL version is >=2.3.0 - which requires C++11. rgeos
>> is
>>> ready for the forthcoming version of GEOS.
>>>
>>> The CRAN team has also been very supportive of our efforts to bring
>>> compiled code in these packages into rchk compliance.
>>>
>>> Please get in touch if you see any loose ends in these releases.
>>>
>>> Roger
>>>
>>>
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
>> http://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
>
>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
http://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en