[R-sig-Geo] CRAN releases of sp, rgdal and rgeos

[Edzer Pebesma]

On 06/12/2018 07:49 PM, Vijay Lulla wrote:
> What is the setup that CRAN uses?  Where can I find this information?

The last two links of https://cran.r-project.org/bin/macosx/ point you
to the libraries it statically links to; further info maybe
https://github.com/r-spatial/sf/issues/327 (or search sf issues for "simon")

> 
> I install most of the R packages on my mac from source and I will be happy
> to help (mostly with answering questions and sharing my notes about how I
> get packages to compile).
> 
> On Tue, Jun 12, 2018 at 12:00 PM, Edzer Pebesma <
> edzer.pebesma using uni-muenster.de> wrote:
> 
>> proj.5 contains new features, but none of these are, or will in the near
>> future be available to R users. All our (= Roger + me) efforts so far
>> have been to get software (rgdal, sf) running with proj.5, not to expose
>> any of its new features.
>> See also https://github.com/r-spatial/sf/issues/545
>>
>> Also: if anyone insists on installing rgdal or sf from source on a Mac
>> but doesn't use the setup CRAN uses for building binaries, neither Roger
>> nor I can assist in case of problems beyond wildly guessing. Of course,
>> any help from anyone else on these kind of issues is highly appreciated.
>>
>> On 06/12/2018 04:41 PM, Vijay Lulla wrote:
>>> Thank you for the explanation of difference between proj.h and proj_api.h
>>> and the recommendation of setting environment variable for pkg-config.  I
>>> had tried using pkg-config by setting env variables but haven't had much
>>> luck getting it to work.
>>> Cordially,
>>> Vijay.
>>>
>>> On Tue, Jun 12, 2018 at 3:11 AM, Roger Bivand <Roger.Bivand using nhh.no>
>> wrote:
>>>
>>>> On Tue, 12 Jun 2018, Vijay Lulla wrote:
>>>>
>>>> I too use R on a mac but I cannot get rgdal to use proj5 (port info
>> proj).
>>>>> rgdal still uses proj4 even when there is a newer version of proj
>> already
>>>>> on my computer.  Neither of these proj packages (proj4 or proj) use
>>>>> pkg-config mechanism even though there are proj.pc files in both
>> versions
>>>>> of the package.  Maybe using the pkg-config mechanism to detect
>> version of
>>>>> packages during source installation might help?  I don't know whether
>> this
>>>>> is a "proj" or a "macports" issue though.
>>>>>
>>>>>
>>>> Neither sp/rgdal nor sf plan to accomodate the proj.h interface until
>> the
>>>> GDAL barn-raising project is completed. Even then, most software using
>> proj
>>>> will have to support the proj_api.h and the proj.h APIs because many
>>>> platforms are bound to move at the speed of the slowest package using
>> proj.
>>>> The proj_api.h API does use the new mechanisms internally, but it isn't
>>>> obvious how or whether proj.h pipelines should be exposed (GRASS has a
>>>> prototype implementation).
>>>>
>>>> If you work in geodetics and need sub-cm precision, this may matter; in
>>>> which case please follow the GDAL barn-raising project and provide use
>>>> cases where the proj.h API is required. The main difference is that the
>>>> proj_api.h API always transforms to the WGS84 datum first then
>> transforms
>>>> to the target datum (many to one then one to many). When this was chosen
>>>> 1984 was not so long ago; the new proj.h API suggests transforming
>> along a
>>>> pipeline from source to target datum without going through WGS84 (many
>> to
>>>> many).
>>>>
>>>> If your question is about how to install rgdal from source on OSX, you
>> may
>>>> need to ensure that you only have one version of proj installed, or use
>>>> configure arguments and load path modifications on your own
>> responsibility.
>>>>
>>>> From rgdal 1.3-1, the pkg-config mechanism is used if the path to
>> proj.pc
>>>> is set correctly (environment variable), but will fall back on standard
>>>> locations if pkg-config proj fails.
>>>>
>>>> Roger
>>>>
>>>>
>>>>
>>>>> On Mon, Jun 11, 2018 at 7:54 PM, Thiago V. dos Santos via R-sig-Geo <
>>>>> r-sig-geo using r-project.org> wrote:
>>>>>
>>>>> Hi Roger, thanks for the follow up. Answering to your comments:
>>>>>>
>>>>>> a) I am using Macports's gdal version 2.3.0.20180523. I am pretty sure
>>>>>> that its source code (i.e. the one that is compiled on my machine) is
>>>>>> downloaded from OSGEO (at least according to https://github.com/
>>>>>> macports/macports-ports/blob/master/gis/gdal/Portfile);
>>>>>>
>>>>>> b) I do have more than one libproj installed on my system (as QGIS for
>>>>>> macOS requires its own gdal and proj stuff), but only the Macports
>> one is
>>>>>> on my path;
>>>>>>
>>>>>>
>>>>>> c) I tried running:
>>>>>> tools::testInstalledPackage("rgdal", outDir=tempdir())list.files(
>>>>>> tempdir())file.show(file.path(tempdir(), "rgdal-Ex.Rout"))
>>>>>>
>>>>>> but it unexpectedly returned "Error in library("rgdal") : there is no
>>>>>> package called ‘rgdal’. Execution halted".
>>>>>>
>>>>>> It looks weird but so far I am able to normally load and use at least
>> a
>>>>>> couple of rgdal's functions.
>>>>>>
>>>>>> I will keep monitoring it though, and report any other issue.
>>>>>>
>>>>>> Cheers,
>>>>>>  -- Thiago V. dos Santos
>>>>>> Postdoctoral Research FellowDepartment of Climate and Space Science
>> and
>>>>>> EngineeringUniversity of Michigan
>>>>>>
>>>>>>     On Sunday, June 10, 2018, 4:46:04 AM EDT, Roger Bivand <
>>>>>> Roger.Bivand using nhh.no> wrote:
>>>>>>
>>>>>>  Thanks for reporting, comments inline below (note that I have no OSX
>>>>>> access at all):
>>>>>>
>>>>>> On Sat, 9 Jun 2018, Thiago V. dos Santos wrote:
>>>>>>
>>>>>> Dear Roger,
>>>>>>> Thank you very much for the excellent work done with those packages.
>>>>>>> Today I update both rgeos and rgdal on my system - macOS 10.13.5 with
>>>>>>> all dependencies installed via MacPorts. I had updated sp a few days
>>>>>>> earlier. The dependencies versions on my system are gdal
>>>>>>> @2.3.0.20180523_0+grib+hdf4+hdf5+jasper+mpich+netcdf and proj
>> @5.1.0_0.
>>>>>>> Rgeos's update went flawlessly, but I got a few errors while updating
>>>>>>> rgdal. It still compiled successfully, but I am concerned that some
>>>>>>> functionality might be compromised due to the errors.
>>>>>>> This is what I got (I have to manually specify the location of proj
>> or
>>>>>>> rgdal won't find it):> install.packages('rgdal', type = "source",
>>>>>>> configure.args=c(
>>>>>>> +    '--with-proj-include=/opt/local/lib/proj5/include',
>>>>>>> +    '--with-proj-lib=/opt/local/lib/proj5/lib'))
>>>>>>> Installing package into ‘/Users/thiago/Documents/R-packages’
>>>>>>> (as ‘lib’ is unspecified)
>>>>>>> trying URL 'https://cran.r-project.org/
>> src/contrib/rgdal_1.3-2.tar.gz'
>>>>>>> Content type 'application/x-gzip' length 1667049 bytes (1.6 MB)
>>>>>>> ==================================================
>>>>>>> downloaded 1.6 MB
>>>>>>>
>>>>>>> * installing *source* package ‘rgdal’ ...
>>>>>>> ** package ‘rgdal’ successfully unpacked and MD5 sums checked
>>>>>>> configure: CC: /usr/bin/clang
>>>>>>> configure: CXX: /usr/bin/clang++
>>>>>>> configure: rgdal: 1.3-2
>>>>>>> checking for /usr/bin/svnversion... yes
>>>>>>> configure: svn revision: 755
>>>>>>> checking whether the C++ compiler works... yes
>>>>>>> checking for C++ compiler default output file name... a.out
>>>>>>> checking for suffix of executables...
>>>>>>> checking whether we are cross compiling... no
>>>>>>> checking for suffix of object files... o
>>>>>>> checking whether we are using the GNU C++ compiler... yes
>>>>>>> checking whether /usr/bin/clang++ accepts -g... yes
>>>>>>> checking whether /usr/bin/clang++ supports C++11 features by
>> default...
>>>>>>>
>>>>>> no
>>>>>>
>>>>>>> checking whether /usr/bin/clang++ supports C++11 features with
>>>>>>>
>>>>>> -std=gnu++11... yes
>>>>>>
>>>>>>> configure: C++11 support available
>>>>>>> checking for gdal-config... /opt/local/bin/gdal-config
>>>>>>> checking gdal-config usability... yes
>>>>>>> configure: GDAL: 2.4.0
>>>>>>>
>>>>>>
>>>>>> All OK up to the GDAL version returned by gdal-config - are you using
>> the
>>>>>> released GDAL 2.3.0 (probably not) or master?
>>>>>>
>>>>>> checking C++11 support for GDAL >= 2.3.0... yes
>>>>>>> checking GDAL version >= 1.11.4... yes
>>>>>>> checking gdal: linking with --libs only... yes
>>>>>>> checking GDAL: /opt/local/share/gdal/pcs.csv readable... yes
>>>>>>> checking proj_api.h presence and usability... yes
>>>>>>> ./configure: line 3395: test: =: unary operator expected
>>>>>>>
>>>>>>
>>>>>> Will check, that line is:
>>>>>>
>>>>>> if test ${PROJ_VERSION} = "" ; then
>>>>>>
>>>>>> from configure.ac line 305. Possibly a shell dialect issue.
>>>>>>
>>>>>> checking PROJ version >= 4.8.0... yes
>>>>>>> checking projects.h presence and usability... yes
>>>>>>>
>>>>>>
>>>>>> These relate to configure.ac lines 376-419, and the outcome: epsg
>> found
>>>>>> and readable is OK - could there be two libproj on your system (maybe
>> for
>>>>>> different architectures)?
>>>>>>
>>>>>> Undefined symbols for architecture x86_64:
>>>>>>>  "_pj_ctx_fclose", referenced from:
>>>>>>>      _main in proj_conf_test2-06fe7d.o
>>>>>>>  "_pj_get_default_ctx", referenced from:
>>>>>>>      _main in proj_conf_test2-06fe7d.o
>>>>>>>  "_pj_open_lib", referenced from:
>>>>>>>      _main in proj_conf_test2-06fe7d.o
>>>>>>> ld: symbol(s) not found for architecture x86_64
>>>>>>> clang: error: linker command failed with exit code 1 (use -v to see
>>>>>>>
>>>>>> invocation)
>>>>>>
>>>>>>> ./configure: line 3511: ./proj_conf_test2: No such file or directory
>>>>>>> checking PROJ.4: epsg found and readable... yes
>>>>>>>
>>>>>>
>>>>>> Same here for next block in configure.ac; conus found and readable.
>>>>>>
>>>>>> Undefined symbols for architecture x86_64:
>>>>>>>  "_pj_ctx_fclose", referenced from:
>>>>>>>      _main in proj_conf_test3-3b7aa2.o
>>>>>>>  "_pj_get_default_ctx", referenced from:
>>>>>>>      _main in proj_conf_test3-3b7aa2.o
>>>>>>>  "_pj_open_lib", referenced from:
>>>>>>>      _main in proj_conf_test3-3b7aa2.o
>>>>>>> ld: symbol(s) not found for architecture x86_64
>>>>>>> clang: error: linker command failed with exit code 1 (use -v to see
>>>>>>>
>>>>>> invocation)
>>>>>>
>>>>>>> ./configure: line 3570: ./proj_conf_test3: No such file or directory
>>>>>>> checking PROJ.4: conus found and readable... yes
>>>>>>> configure: Package CPP flags:  -I/opt/local/include
>>>>>>>
>>>>>> -I/opt/local/lib/proj5/include
>>>>>>
>>>>>>> configure: Package LIBS:  -L/opt/local/lib -lgdal
>>>>>>>
>>>>>> -L/opt/local/lib/proj5/lib -lproj
>>>>>>
>>>>>>> configure: creating ./config.status
>>>>>>> config.status: creating src/Makevars
>>>>>>> ** libs
>>>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>>>> work/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2 -c OGR_write.cpp -o OGR_write.o
>>>>>>
>>>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>>>> work/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2 -c gdal-bindings.cpp -o gdal-bindings.o
>>>>>>
>>>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2  -c init.c -o init.o
>>>>>>
>>>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2  -c inverser.c -o inverser.o
>>>>>>
>>>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2  -c local_stubs.c -o local_stubs.o
>>>>>>
>>>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>>>> work/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2 -c ogr_geom.cpp -o ogr_geom.o
>>>>>>
>>>>>>> /usr/bin/clang -I"/Library/Frameworks/R.framework/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2  -c ogr_polygons.c -o ogr_polygons.o
>>>>>>
>>>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>>>> work/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2 -c ogr_proj.cpp -o ogr_proj.o
>>>>>>
>>>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>>>> work/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2 -c ogrdrivers.cpp -o ogrdrivers.o
>>>>>>
>>>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>>>> work/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2 -c ogrsource.cpp -o ogrsource.o
>>>>>>
>>>>>>> clang++ -std=gnu++11 -I"/Library/Frameworks/R.frame
>>>>>>> work/Resources/include"
>>>>>>>
>>>>>> -DNDEBUG -I/opt/local/include -I/opt/local/lib/proj5/include
>>>>>> -I"/Users/thiago/Documents/R-packages/sp/include"
>> -I/usr/local/include
>>>>>> -fPIC  -Wall -g -O2 -c projectit.cpp -o projectit.o
>>>>>>
>>>>>>> clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names
>>>>>>>
>>>>>> -undefined dynamic_lookup -single_module -multiply_defined suppress
>>>>>> -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o
>>>>>> rgdal.so OGR_write.o gdal-bindings.o init.o inverser.o local_stubs.o
>>>>>> ogr_geom.o ogr_polygons.o ogr_proj.o ogrdrivers.o ogrsource.o
>> projectit.o
>>>>>> -L/opt/local/lib -lgdal -L/opt/local/lib/proj5/lib -lproj
>>>>>> -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework
>>>>>> -Wl,CoreFoundation
>>>>>>
>>>>>>> installing to /Users/thiago/Documents/R-packages/rgdal/libs
>>>>>>> ** R
>>>>>>> ** data
>>>>>>> ** inst
>>>>>>> ** byte-compile and prepare package for lazy loading
>>>>>>> ** help
>>>>>>> *** installing help indices
>>>>>>> ** building package indices
>>>>>>> ** installing vignettes
>>>>>>> ** testing if installed package can be loaded
>>>>>>> * DONE (rgdal)
>>>>>>>
>>>>>>> The downloaded source packages are in
>>>>>>>
>>>>>>> ‘/private/var/folders/_z/01gg71zs19g816v6m2dddt8w0000gn
>>>>>>>
>>>>>> /T/Rtmp5K6lt8/downloaded_packages’
>>>>>>
>>>>>>> ​Is this something that needs to be investigated, or I can just
>> safely
>>>>>>> ignore those messages?
>>>>>>>
>>>>>>
>>>>>> Please try to run:
>>>>>>
>>>>>> tools::testInstalledPackage("rgdal", outDir=tempdir())
>>>>>> list.files(tempdir())
>>>>>> file.show(file.path(tempdir(), "rgdal-Ex.Rout"))
>>>>>>
>>>>>> and if the outcomes are as expected, you should be OK, please let us
>>>>>> know.
>>>>>>
>>>>>> If any OSX users installing from source can contribute, that would be
>>>>>> useful. It will take a little while before the CRAN OSX binaries
>> catch up
>>>>>> with new PROJ and GDAL.
>>>>>>
>>>>>> Again thanks for reporting!
>>>>>>
>>>>>> Roger
>>>>>>
>>>>>> Many thanks, -- Thiago V. dos Santos
>>>>>>> Postdoctoral Research FellowDepartment of Climate and Space Science
>> and
>>>>>>>
>>>>>> EngineeringUniversity of Michigan
>>>>>>
>>>>>>>
>>>>>>>    On Friday, June 8, 2018, 2:15:43 PM EDT, Roger Bivand <
>>>>>>>
>>>>>> Roger.Bivand using nhh.no> wrote:
>>>>>>
>>>>>>>
>>>>>>> There are new releases of sp, rgdal and rgeos on CRAN. Please
>> install sp
>>>>>>> first, then the other two, which link to the installed sp. They all
>>>>>>> address so-called rchk issues, which have not so far been a problem,
>> but
>>>>>>> might have become more fragile as R's internal memory management is
>> made
>>>>>>> even more efficient. This involves compiled code using memory
>> allocated
>>>>>>>
>>>>>> by
>>>>>>
>>>>>>> R to be freed by R's garbage collector, which has to know if an
>> object
>>>>>>> is
>>>>>>> still being used. Tomas Kalibera, the author of rchk, helped resolve
>> and
>>>>>>> explain the issues encountered - what was good coding practice
>> fifteen
>>>>>>> years ago isn't always still good practice.
>>>>>>>
>>>>>>> In addition, the earliest versions of GDAL and PROJ with which rgdal
>>>>>>> will
>>>>>>> work have been updated, and set to PROJ 4.8.0 and GDAL 1.11.4. The
>>>>>>>
>>>>>> current
>>>>>>
>>>>>>> released versions of PROJ and GDAL are to be prefered, as bugs have
>> been
>>>>>>> fixed and new features and drivers introduced. A check has been put
>>>>>>> in place to trap attempts to install rgdal without a C++11-capable
>>>>>>> compiler when the GDAL version is >=2.3.0 - which requires C++11.
>> rgeos
>>>>>>>
>>>>>> is
>>>>>>
>>>>>>> ready for the forthcoming version of GEOS.
>>>>>>>
>>>>>>> The CRAN team has also been very supportive of our efforts to bring
>>>>>>> compiled code in these packages into rchk compliance.
>>>>>>>
>>>>>>> Please get in touch if you see any loose ends in these releases.
>>>>>>>
>>>>>>> Roger
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Roger Bivand
>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>> voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
>>>>>> http://orcid.org/0000-0003-2392-6140
>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>>         [[alternative HTML version deleted]]
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-Geo mailing list
>>>>>> R-sig-Geo using r-project.org
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> --
>>>> Roger Bivand
>>>> Department of Economics, Norwegian School of Economics,
>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>> voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
>>>> http://orcid.org/0000-0003-2392-6140
>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>
>>>
>>>
>>>
>>
>> --
>> Edzer Pebesma
>> Institute for Geoinformatics
>> Heisenbergstrasse 2, 48151 Muenster, Germany
>> Phone: +49 251 8333081
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
> 
> 
> 

-- 
Edzer Pebesma
Institute for Geoinformatics
Heisenbergstrasse 2, 48151 Muenster, Germany
Phone: +49 251 8333081