[ESS] ess-17.11-tgz is not compressed

[Martin Maechler]

>>>>> Ulrich Mueller <ulm at gentoo.org>
>>>>>     on Fri, 8 Dec 2017 09:34:52 +0100 writes:

>>>>> On Thu, 7 Dec 2017, Martin Maechler wrote:
    >> well, I'm managing the ess.r-project.org server and I
    >> have actually mounted the file system where ess-17.11.tgz
    >> lives and 'ls -l' and 'file' say

    >> ls -l ess-17.11.tgz -rw-r--r--. 1 maechler sfsstaff
    >> 3275703 Nov 13 15:13 ess-17.11.tgz

    >> file ess-17.11.tgz ess-17.11.tgz: gzip compressed data,
    >> last modified: Mon Nov 13 14:13:29 2017, from Unix

    > So ess-17.11.tgz is what we distro builders would call the
    > "pristine tarball". It is the file that should be stored
    > after downloading and whose integrity is verified with a
    > checksum [1].

    >> Now look closely at the output of wget -- which I can
    >> reproduce in my (Fedora 26) Linux :

    >>> ~/tmp/ess $ wget
    >>> http://ess.r-project.org/downloads/ess/ess-17.11.tgz
    >>> --2017-12-05 08:12:09--
    >>> http://ess.r-project.org/downloads/ess/ess-17.11.tgz
    >>> Resolving ess.r-project.org... 129.132.119.195
    >>> Connecting to
    >>> ess.r-project.org|129.132.119.195|:80... connected.
    >>> HTTP request sent, awaiting response... 200 OK Length:
    >>> 3275703 (3.1M) [application/x-tar]
    >> ^^^^^^^^^^^^^^
    >>> Saving to: ‘ess-17.11.tgz’
    >>> 
    >>> ess-17.11.tgz
    >>> 100%[===========================================================================================>]
    >>> 3.12M 6.30MB/s in 0.5s
    >>> 
    >>> 2017-12-05 08:12:09 (6.30 MB/s) - ‘ess-17.11.tgz’ saved
    >>> [8898560]
    >> ^^^^^^^^

    > Regardless of wget's choice of defaults (which is a
    > separate discussion), I believe that the server's response
    > is not correct:

    > | $ wget --server-response
    > http://ess.r-project.org/downloads/ess/ess-17.11.tgz |
    > --2017-12-08 09:07:29--
    > http://ess.r-project.org/downloads/ess/ess-17.11.tgz |
    > Resolving ess.r-project.org... 129.132.119.195 |
    > Connecting to
    > ess.r-project.org|129.132.119.195|:80... connected.  |
    > HTTP request sent, awaiting response...  | HTTP/1.1 200 OK
    > | Date: Fri, 08 Dec 2017 08:07:29 GMT | Server:
    > Apache/2.2.32 (Unix) | Last-Modified: Mon, 13 Nov 2017
    > 14:13:29 GMT | ETag: "3e16cc-31fbb7-55ddddfe47440" |
    > Accept-Ranges: bytes | Content-Length: 3275703 |
    > Keep-Alive: timeout=15, max=100 | Connection: Keep-Alive |
    > Content-Type: application/x-tar | Content-Encoding: gzip |
    > Length: 3275703 (3.1M) [application/x-tar] | Saving to:
    > ‘ess-17.11.tgz’ | [...]  | 2017-12-08 09:07:34 (714 KB/s)
    > - ‘ess-17.11.tgz’ saved [8898560]

    > The server is sending a "Content-Encoding: gzip" header,
    > indicating that gzip is used for transport compression,
    > and that the compression should be undone on the
    > receiver's side. But see above, we consider ess-17.11.tgz
    > as the pristine sources, not ess-17.11.tar. So the server
    > should neither apply nor indicate any Content-Encoding for
    > it.

    >> so indeed wget seems to do __silly!__ behave a bit
    >> magically nowadays ... -- at least being honest about it:

    >> It gets a *.tgz of size 3275703 bytes (~ 3.1 M) and
    >> internally uses gunzip absolutely with*OUT* saying so,
    >> but then honestly reports that the result is of size
    >> 8898560 bytes

    > No. The client is explicitly asking for a .tgz file, but
    > the server indicates in its response that it is sending a
    > tar file (incorrectly named *.tgz, on top of that) which
    > is compressed only for the purpose of transport.

    > I don't want to clutter this message with another log, but
    > in the downstream bug report [2] you can see that
    > downloading the file from one of the Gentoo mirrors
    > behaves correctly.


    > [1] https://gitweb.gentoo.org/repo/gentoo.git/tree/app-emacs/ess/Manifest
    > [2] https://bugs.gentoo.org/639752#c8

Thank you, Ulrich.

Your reasoning seems quite convincing,  and I am enquiring why
our server does what you show above it does, and how we can
change that.

Martin

--
Martin <Maechler at stat.math.ethz.ch>   http://stat.ethz.ch/~maechler
Seminar für Statistik, ETH Zürich   HG G 16         Rämistrasse 101
CH-8092 Zurich, SWITZERLAND
phone: +41-44-632-3408      fax: ...-1228     <><