[Rd] gfortran 9 quantreg bug

Koenker, Roger W rkoenker @end|ng |rom ||||no|@@edu
Mon Aug 5 17:19:15 CEST 2019 

With extensive help from Dirk Eddelbuettel I have installed 
docker on my mac mini from 

    https://hub.docker.com/editions/community/docker-ce-desktop-mac

which installs from a dmg in quite standard fashion.  This has allowed
me to simulate running R in a Debian environment with gfortran-9 and
begin the process of debugging my ancient rqbr.f code.

Some further details:

0.  After some initial testing, e.g.

    docker --version 
    docker run hello-world       

1.  Download r-base and test os

    docker pull r-base                       $ downloads r-base for us 
    docker run --rm -ti r-base R --version   # to check we have the R we want
    docker run --rm -ti r-base bash          # now in shell, Ctrl-d to exit

2.  Setup working directory -- tell Docker to run from the current directory
    and access

    cd projects/rq
    docker run --rm -ti -v ${PWD}:/work -w /work r-base bash

    This put the contents of projects/rq into the /work directory.

    root using 90521904fa86:/work# apt-get update
    Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [149 kB]
    Get:2 http://cdn-fastly.deb.debian.org/debian testing InRelease [117 kB]
    Get:3 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8,385 kB]
    Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 Packages [7,916 kB]
    Fetched 16.6 MB in 4s (4,411 kB/s)                           
    Reading package lists... Done
  
3.  Get gcc-9 and gfortran-9

    root using 90521904fa86:/work# apt-get install gcc-9 gfortran-9
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    The following additional packages will be installed:
    cpp-9 gcc-9-base libasan5 libatomic1 libcc1-0 libgcc-9-dev libgcc1 libgfortran-9-dev
    libgfortran5 libgomp1 libitm1 liblsan0 libquadmath0 libstdc++6 libtsan0 libubsan1
    Suggested packages:
    gcc-9-locales gcc-9-multilib gcc-9-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg
    libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libquadmath0-dbg gfortran-9-multilib
    gfortran-9-doc libgfortran5-dbg libcoarrays-dev
    The following NEW packages will be installed:
    cpp-9 gcc-9 gfortran-9 libgcc-9-dev libgfortran-9-dev
    The following packages will be upgraded:
    gcc-9-base libasan5 libatomic1 libcc1-0 libgcc1 libgfortran5 libgomp1 libitm1 liblsan0
    libquadmath0 libstdc++6 libtsan0 libubsan1
    13 upgraded, 5 newly installed, 0 to remove and 71 not upgraded.
    Need to get 35.6 MB of archives.
    After this operation, 107 MB of additional disk space will be used.
    Do you want to continue? [Y/n] Y
    Get:1 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libasan5 amd64 9.1.0-10 [390 kB]
    Get:2 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libubsan1 amd64 9.1.0-10 [128 kB]
    Get:3 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libtsan0 amd64 9.1.0-10 [295 kB]
    Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gcc-9-base amd64 9.1.0-10 [190 kB]
    Get:5 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libstdc++6 amd64 9.1.0-10 [500 kB]
    Get:6 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libquadmath0 amd64 9.1.0-10 [145 kB]
    Get:7 http://cdn-fastly.deb.debian.org/debian testing/main amd64 liblsan0 amd64 9.1.0-10 [137 kB]
    Get:8 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libitm1 amd64 9.1.0-10 [27.6 kB]
    Get:9 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgomp1 amd64 9.1.0-10 [88.1 kB]
    Get:10 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgfortran5 amd64 9.1.0-10 [633 kB]
    Get:11 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libcc1-0 amd64 9.1.0-10 [47.7 kB]
    Get:12 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libatomic1 amd64 9.1.0-10 [9,012 B]
    Get:13 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgcc1 amd64 1:9.1.0-10 [40.5 kB]
    Get:14 http://cdn-fastly.deb.debian.org/debian testing/main amd64 cpp-9 amd64 9.1.0-10 [9,667 kB]
    Get:15 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgcc-9-dev amd64 9.1.0-10 [2,346 kB]
    Get:16 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gcc-9 amd64 9.1.0-10 [9,945 kB]
    Get:17 http://cdn-fastly.deb.debian.org/debian testing/main amd64 libgfortran-9-dev amd64 9.1.0-10 [676 kB]
    Get:18 http://cdn-fastly.deb.debian.org/debian testing/main amd64 gfortran-9 amd64 9.1.0-10 [10.4 MB]
    Fetched 35.6 MB in 6s (6,216 kB/s)      
    debconf: delaying package configuration, since apt-utils is not installed
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../libasan5_9.1.0-10_amd64.deb ...
    Unpacking libasan5:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../libubsan1_9.1.0-10_amd64.deb ...
    Unpacking libubsan1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../libtsan0_9.1.0-10_amd64.deb ...
    Unpacking libtsan0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../gcc-9-base_9.1.0-10_amd64.deb ...
    Unpacking gcc-9-base:amd64 (9.1.0-10) over (9.1.0-8) ...
    Setting up gcc-9-base:amd64 (9.1.0-10) ...
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../libstdc++6_9.1.0-10_amd64.deb ...
    Unpacking libstdc++6:amd64 (9.1.0-10) over (9.1.0-8) ...
    Setting up libstdc++6:amd64 (9.1.0-10) ...
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../0-libquadmath0_9.1.0-10_amd64.deb ...
    Unpacking libquadmath0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../1-liblsan0_9.1.0-10_amd64.deb ...
    Unpacking liblsan0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../2-libitm1_9.1.0-10_amd64.deb ...
    Unpacking libitm1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../3-libgomp1_9.1.0-10_amd64.deb ...
    Unpacking libgomp1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../4-libgfortran5_9.1.0-10_amd64.deb ...
    Unpacking libgfortran5:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../5-libcc1-0_9.1.0-10_amd64.deb ...
    Unpacking libcc1-0:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../6-libatomic1_9.1.0-10_amd64.deb ...
    Unpacking libatomic1:amd64 (9.1.0-10) over (9.1.0-8) ...
    Preparing to unpack .../7-libgcc1_1%3a9.1.0-10_amd64.deb ...
    Unpacking libgcc1:amd64 (1:9.1.0-10) over (1:9.1.0-8) ...
    Setting up libgcc1:amd64 (1:9.1.0-10) ...
    Selecting previously unselected package cpp-9.
    (Reading database ... 17787 files and directories currently installed.)
    Preparing to unpack .../cpp-9_9.1.0-10_amd64.deb ...
    Unpacking cpp-9 (9.1.0-10) ...
    Selecting previously unselected package libgcc-9-dev:amd64.
    Preparing to unpack .../libgcc-9-dev_9.1.0-10_amd64.deb ...
    Unpacking libgcc-9-dev:amd64 (9.1.0-10) ...
    Selecting previously unselected package gcc-9.
    Preparing to unpack .../gcc-9_9.1.0-10_amd64.deb ...
    Unpacking gcc-9 (9.1.0-10) ...
    Selecting previously unselected package libgfortran-9-dev:amd64.
    Preparing to unpack .../libgfortran-9-dev_9.1.0-10_amd64.deb ...
    Unpacking libgfortran-9-dev:amd64 (9.1.0-10) ...
    Selecting previously unselected package gfortran-9.
    Preparing to unpack .../gfortran-9_9.1.0-10_amd64.deb ...
    Unpacking gfortran-9 (9.1.0-10) ...
    Setting up libgomp1:amd64 (9.1.0-10) ...
    Setting up libasan5:amd64 (9.1.0-10) ...
    Setting up libquadmath0:amd64 (9.1.0-10) ...
    Setting up libatomic1:amd64 (9.1.0-10) ...
    Setting up libgfortran5:amd64 (9.1.0-10) ...
    Setting up libubsan1:amd64 (9.1.0-10) ...
    Setting up cpp-9 (9.1.0-10) ...
    Setting up libcc1-0:amd64 (9.1.0-10) ...
    Setting up liblsan0:amd64 (9.1.0-10) ...
    Setting up libitm1:amd64 (9.1.0-10) ...
    Setting up libtsan0:amd64 (9.1.0-10) ...
    Setting up libgcc-9-dev:amd64 (9.1.0-10) ...
    Setting up gcc-9 (9.1.0-10) ...
    Setting up libgfortran-9-dev:amd64 (9.1.0-10) ...
    Setting up gfortran-9 (9.1.0-10) ...
    Processing triggers for libc-bin (2.28-10) ...
    root using 90521904fa86:/work# pwd
    
4.  At this point I removed some dependencies from the package
    quantreg that I knew were not relevant to the debugging problem at
    hand. This included an attempt to set compiler flags in
    quantreg/src/Makevars, but this didn't work.

5.  Set compiler flags as follows:

    root using 90521904fa86:/work# mkdir ~/.R; vi ~/.R/Makevars 

	CC=gcc-9
	FC=gfortran-9
	F77=gfortran-9

    Alternatively, one can find the settings of CC, FC, CXX, ... in /etc/R/Makeconf
    and alter them there.
    
6.  At this point 

    R CMD INSTALL quantreg_5.43.tar.gz

    did use the gfortran-9  compiler and this version did reproduce the 
    error initially reported by Kurt and Brian.  

7.  Now it is (just!) a matter of finding the bug.
     
Roger Koenker
r.koenker using ucl.ac.uk
Department of Economics, UCL
London  WC1H 0AX.


> On Aug 4, 2019, at 3:41 PM, Dirk Eddelbuettel <edd using debian.org> wrote:
> 
> 
> Roger,
> 
> On 4 August 2019 at 06:48, Koenker, Roger W wrote:
> | I’d like to solicit some advice on a debugging problem I have in the quantreg package.
> | Kurt and Brian have reported to me that on Debian machines with gfortran 9
> | 
> | library(quantreg)
> | f = summary(rq(foodexp ~ income, data = engel, tau = 1:4/5))
> | plot(f)
> | 
> | fails because summary() produces bogus estimates of the coefficient bounds.
> | This example has been around in my R package from the earliest days of R, and
> | before that in various incarnations of S.  The culprit is apparently rqbr.f which is
> | even more ancient, but must have something that gfortran 9 doesn’t approve of.
> | 
> | I note that in R-devel there have been some other issues with gfortran 9, but these seem
> | unrelated to my problem.  Not having access to a machine with an R/gfortran9
> | configuration, I can’t  apply my rudimentary debugging methods.  I’ve considered
> | trying to build gfortran on my mac air and then building R from source, but before
> | going down this road, I wondered whether others had other suggestions, or
> | advice about  my proposed route.  As far as I can see there are not yet
> | binaries for gfortran 9 for osx.
> 
> Maybe installing and running Docker on your mac is an alternative?
> 
> Minimally viable example using
> 
>  a) docker (on Linux, but it is portable) and
> 
>  b) the current official 'r-base' container (an alias to our Rocker r-base container)
> 
> r-base is begged to Debian testing, and also allows you to get Debian
> unstable.  Below I fire up the container, tell it to use bash (not R) and update
> 
>  edd using rob:~/git$ docker run --rm -ti r-base bash
>  root using 1307193fadf4:/# 
>  root using 1307193fadf4:/# apt-get update
>  Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [149 kB]
>  Get:2 http://cdn-fastly.deb.debian.org/debian testing InRelease [117 kB]
>  Get:3 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8,385 kB]
>  Get:4 http://cdn-fastly.deb.debian.org/debian testing/main amd64 Packages [7,918 kB]
>  Fetched 16.6 MB in 4s (4,649 kB/s)                           
>  Reading package lists... Done
>  root using 1307193fadf4:/# apt-cache policy gcc-9
>  gcc-9:
>    Installed: (none)
>    Candidate: 9.1.0-10
>    Version table:
>       9.1.0-10 990
>          990 http://deb.debian.org/debian testing/main amd64 Packages
>          500 http://http.debian.net/debian sid/main amd64 Packages
>  root using 1307193fadf4:/# apt-cache policy gfortran-9
>  gfortran-9:
>    Installed: (none)
>    Candidate: 9.1.0-10
>    Version table:
>       9.1.0-10 990
>          990 http://deb.debian.org/debian testing/main amd64 Packages
>          500 http://http.debian.net/debian sid/main amd64 Packages
>  root using 1307193fadf4:/# 
> 
> At this point it just a matter of actually installing gcc-9 and gfortran-9
> (via apt-get install ...), and setting CC, FC, F77 and whichever other
> environment variables the R build reflect to build quantreg.
> 
> That said, this will be Debian's standard gfortran-9.  What is at times a
> little frustrating is that some of the builds used by some of the CRAN tests
> use local modifications which make their behaviour a little harder to
> reproduce.  I have an open issue with my (also old and stable) digest package
> which goes belly-up on a clang-on-Fedora build and nowhere else -- and I have
> been unable to reproduce this too.
> 
> For such cases, having Docker container would be one possible way of
> giving access to the test environment to make it accessible to more users.
> 
> Best,  Dirk
> 
> 
> | 
> | Thanks,
> | Roger
> | 
> | Roger Koenker
> | r.koenker using ucl.ac.uk<mailto:r.koenker using ucl.ac.uk>
> | Department of Economics, UCL
> | London  WC1H 0AX.
> | 
> | 
> | 
> | 	[[alternative HTML version deleted]]
> | 
> | ______________________________________________
> | R-devel using r-project.org mailing list
> | https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> -- 
> http://dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org

More information about the R-devel mailing list