[R-sig-Geo] Pre-GDAL 2: rgdal changes - from all according to their means

Wed Jun 3 15:08:37 CEST 2015

In case problems persist, a copy of the source tarball is at:

http://spatial.nhh.no/R/rgdal/rgdal_1.0-2.tar.gz

However, SVN access is easier as updating the local copy is simpler than 
downloading again, if we need to interact in fixing bugs.

Roger

On Wed, 3 Jun 2015, chris english wrote:

> Successful linux download an hour ago.
>
> Chris
>
> On Wed, Jun 3, 2015 at 8:33 AM, Frede Aakmann Tøgersen <frtog at vestas.com>
> wrote:
>
>> Hi
>>
>> Anyone else having problem with checking out the repos?
>>
>> On my Windows box:
>>
>> Command: Checkout from svn://scm.r-forge.r-project.org/svnroot/rgdal/pkg,
>> revision HEAD, Fully recursive, Externals included
>> Error: Unable to connect to a repository at URL
>> Error:  'svn://scm.r-forge.r-project.org/svnroot/rgdal/pkg'
>> Error: Can't connect to host 'scm.r-forge.r-project.org': No connection
>> could be made
>> Error:  because the target machine actively refused it.
>> Completed!:
>>
>>
>> On our cluster:
>>
>> [frtog at dkrdsfshn2 ~]$ svn checkout svn://
>> scm.r-forge.r-project.org/svnroot/rgdal/pkg rgdal
>> svn: Can't connect to host 'scm.r-forge.r-project.org': Connection refused
>> [frtog at dkrdsfshn2 ~]$
>>
>> So no anonymous access?
>>
>>
>> Yours sincerely / Med venlig hilsen
>>
>> Frede Aakmann Tøgersen
>> Specialist, M.Sc., Ph.D.
>> Plant Performance & Modeling
>>
>> Technology & Service Solutions
>> T +45 9730 5135
>> M +45 2547 6050
>> frtog at vestas.com
>> http://www.vestas.com
>>
>> Company reg. name: Vestas Wind Systems A/S
>> This e-mail is subject to our e-mail disclaimer statement.
>> Please refer to www.vestas.com/legal/notice
>> If you have received this e-mail in error please contact the sender.
>>
>>
>>
>> -----Original Message-----
>> From: R-sig-Geo [mailto:r-sig-geo-bounces at r-project.org] On Behalf Of
>> Roger Bivand
>> Sent: 3. juni 2015 11:21
>> To: Michael Sumner
>> Cc: r-sig-geo
>> Subject: Re: [R-sig-Geo] Pre-GDAL 2: rgdal changes - from all according to
>> their means
>>
>> On Wed, 3 Jun 2015, Michael Sumner wrote:
>>
>>> On Wed, 3 Jun 2015 at 17:14 Roger Bivand <Roger.Bivand at nhh.no> wrote:
>>>
>>>> On Tue, 2 Jun 2015, chris english wrote:
>>>>
>>>>> Roger,
>>>>>
>>>>> I second Michael Sumner's excitement. I built a POSTGist SFCGAL stack
>>>> based
>>>>> on GDAL 2.0.0beta, anticipating that I could seamlessly integrate with
>> R
>>>>> and was disappointed that I couldn't use  2.0.0.beta  because at the
>>>> point
>>>>> only sp-1.1-1 was supported.
>>>>
>>>> Chris,
>>>>
>>>> Excitement doesn't help. GDAL 2 is only an advance over GDAL 1 in that
>> it
>>>> makes the code base easier to maintain because only the GDAL driver
>>>> manager is supported, and vector drivers have been folded into the same
>>>> model. Beyond that, we have continuing feature creep (e.g. use of
>>>> Integer64 to hold IDs that notionally should be strings (no arithmetic)
>>>> and which always lose leading zeros, and others. The key challenge
>>>> (threat) to GDAL is the very limited number of core developers, followed
>>>> by the same danger affecting many of GDAL's external dependencies.
>>>>
>>>> General comment: Could someone who knows how to do it please set up a
>>>> searchable scribble sheet to which ordered examples, open issues, ideas,
>>>> etc. could be contributed? This thread is now the prime location, but
>>>> probably will become unwieldy.
>>>>
>>>>
>>> What about MoPad?
>>> https://etherpad.mozilla.org/8bU6hpr0YY
>>
>> OK, thanks, I've added som comments. I've also removed the
>> trunk/tags/branch structure on R-Forge, so
>>
>> svn checkout svn://scm.r-forge.r-project.org/svnroot/rgdal/pkg
>>
>> does a clean checkout that can be updated, but should be put in an rgdal
>> folder to avoid splatter.
>>
>> Roger
>>
>>>
>>> I set up a repo in github too but that might not be of interest since we
>>> need pull-requests, forks etc.
>>>
>>> https://github.com/mdsumner/rgdal_scribblesheet
>>>
>>> Happy to help with any of this in whatever way people prefer.
>>>
>>> Cheers, Mike.
>>>
>>>
>>>
>>>> You will never be able to switch easily between rgdal installed against
>>>> GDAL 1 and rgdal installed against GDAL 2. Any possibilities are only
>>>> available to those installing GDAL from source, then rgdal from source,
>> as
>>>> GDAL 2 overwrites GDAL 1 (same name for gdal-config, same app names) if
>>>> --prefix= is the same. If --prefix= is different between 1 and 2 when
>>>> installing GDAL, and you take care with LD path names and use of
>> ldconfig,
>>>> you can pass these different locations of GDAL 1 and GDAL 2 through when
>>>> installing rgdal. Then you'd also have to remember to install rgdal for
>>>> GDAL 1 in say one R library folder, for GDAL 2 in a different library
>>>> folder, and add the appropriate one to .libPaths(). It would be messy.
>>>>
>>>> Performance-wise, there should be no user-impacting difference between
>>>> GDAL 1 and 2, or between rgdal opening for GDAL 1 or 2, and rgdal (<=
>>>> 0.9-3), which blocks GDAL 2.
>>>>
>>>> The task now is to locate possible differences in behaviour between
>> rgdal
>>>> 0.9-3 with GDAL 1, and rgdal 1.0-2 with either GDAL 1 or 2. If there are
>>>> 1|2 differences, are they because of changes in GDAL, or because rgdal
>> was
>>>> making inappropriate assumptions about GDAL?
>>>>
>>>> So far, we really don't know the Integer64 field type in GDAL 2 vectors
>> is
>>>> going to play out - as of now rgdal 1.0-2 truncates to integer. We don't
>>>> know how the changes in the driver manager for vector dsn and layers
>>>> affect layer and dsn overwriting (I'm seeing very odd OGRErr numbers in
>>>> GDAL 2, and apparently different behaviour for some drivers). Writing
>>>> MapInfo File TAB files looks broken (maybe a dsn creation option needs
>>>> setting, when it wasn't needed before), etc.
>>>>
>>>> Some of this can be fixed by nasty kludges in rgdal, some should be
>> fixed
>>>> in GDAL 2, some are real misunderstandings. I don't have the time to try
>>>> to make an rgdal Windows binary built against Windows GDAL 2 by
>>>> cross-compiling for trying things out under Windows without user
>>>> compilation. The transition will take time, but anyone who can install
>>>> GDAL and rgdal from source, and who is interested and/or needs rgdal to
>>>> work predictably, is welcome to run some standard procedures with rgdal
>>>> 0.9-3 and GDAL 1, storing a text version of key results; install rgdal
>>>> 1.0-2 (or later) and GDAL 1 (or later checked out from R-Forge and
>> living
>>>> under pkg/:
>>>>
>>>> #cd rgdal
>>>> R CMD build pkg
>>>> R CMD check rgdal_1.0-2.tar.gz
>>>> # maybe --install-args='--configure.args=...'
>>>> R CMD INSTALL rgdal_1.0-2.tar.gz
>>>> # maybe --configure.args=...
>>>>
>>>> rerun the procedure and compare the results to see whether changes in
>>>> rgdal have led to changes in the results, the re-install rgdal 1.0-2
>> with
>>>> GDAL 2, rerun the procedure and compare the results, and compare those
>> for
>>>> GDAL 1 and GDAL 2.
>>>>
>>>> In particular, we need confirmation that the database access drivers
>> work
>>>> across these shifts.
>>>>
>>>> Roger
>>>>
>>>>>
>>>>> I have a couple of future projects that are going to fundamentally
>> depend
>>>>> on GDAL and I hope to be able to address them from within R.  As an R
>>>>> beginner I'm happy I have in my notes on how to point to my /opt/gdal(
>>>>> please excuse if generalizing between an installed /opt/gdal(2.0,0.beta
>>>> and
>>>>> /usr/local/gdal(earlier.build)  build, but many people perhaps have not
>>>>> been making a diary of R success(es).
>>>>>
>>>>> In the simplest case it could be a reminder to us beginners how we
>> might
>>>>> load a library(rgdal("in this case 2.0.0.beta vs our normal
>>>>> library(rgdal)).  Well we are beginners, but we still want out stuff to
>>>>> work and we want to contribute to a successful and comprehensive 2.2.0
>>>>> release. So we'll run our particular research examples through 2.0.0
>> and
>>>>> report results, if we know how to load one vs. another library.
>>>>>
>>>>> If intermediate or expert, different examples would be deployed.
>>>>>
>>>>> I apologize in advance if these issues  or methods have already been
>>>>> addressed at SVN and I failed to notice them.  If that is the case I
>> will
>>>>> dig further, and please ignore me.
>>>>>
>>>>> I think GDAL and rgdal are intrinsically important tools to this
>>>> community
>>>>> and everyone wants them to work; please just tell us, by general
>> example,
>>>>> how we might be best of service.
>>>>>
>>>>> I hope the foregoing is comprehensible. My thoughts, and looking
>> forward
>>>> to
>>>>> to contributing in a useful fashion to this important package (as a
>>>>> beginner).
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Chris
>>>>>
>>>>>
>>>>> The second beta of GDAL 2 is now available, and as of revision 535 on
>>>>>> R-Forge, the legacy rgdal package passes R CMD check with either GDAL
>>>>>> 1.11.2 (or earlier) or GDAL 2.0.0 beta 2.
>>>>>>
>>>>>> One or two issues are known (Integer64 in vector fields and FIDs not
>>>>>> supported in R; gdalDrivers() reports both raster and vector drivers;
>>>> the
>>>>>> MapInfo File TAB driver doesn't work for writing, ...), but others
>>>> remain
>>>>>> to be discovered.
>>>>>>
>>>>>> For those who need rgdal in production, and can install the 2.0.0
>> beta,
>>>> it
>>>>>> would be a really good use of time to identify issues now, rather than
>>>> when
>>>>>> GDAL 2 starts to become the standard, stable release. Anyone else
>>>> needing
>>>>>> an itch to scratch is also, of course, welcome to contribute.
>>>>>>
>>>>>> The rgdal package will continue to condition on GDAL 1 or 2, so
>>>> hopefully
>>>>>> those users who do not need to move to GDAL 2 will not be affected.
>>>>>> However, it is worth noting that GDAL is maintained by very, very, few
>>>>>> volunteers (even plural is questionable here), and when they feel that
>>>>>> backporting fixed from GDAL 2 to GDAL 1 is taking time from more
>>>> important
>>>>>> things, you will be stranded with EOL software.
>>>>>>
>>>>>> So please consider taking the time to contribute to the idenfication
>> of
>>>>>> issues in the development version of rgdal built against GDAL 2 and/or
>>>> 1,
>>>>>> available for anonymous SVN checkout at:
>>>>>>
>>>>>> svn checkout svn://scm.r-forge.r-project.org/svnroot/rgdal/trunk
>>>>>>
>>>>>> Enjoy!
>>>>>>
>>>>>> Roger
>>>>>>
>>>>>> --
>>>>>> Roger Bivand
>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>> voice: +47 55 95 93 55; fax +47 55 95 91 00
>>>>>> e-mail: Roger.Bivand at nhh.no
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-Geo mailing list
>>>>>> R-sig-Geo at r-project.org
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Roger Bivand
>>>> Department of Economics, Norwegian School of Economics,
>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>> voice: +47 55 95 93 55; fax +47 55 95 91 00
>>>> e-mail: Roger.Bivand at nhh.no
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>
>>>
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; fax +47 55 95 91 00
>> e-mail: Roger.Bivand at nhh.no
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: Roger.Bivand at nhh.no