[Rd] patch to support custom HTTP headers in download.file() and url()

[Martin Maechler]

>>>>> Gábor Csárdi 
>>>>>     on Tue, 4 Dec 2018 21:23:20 +0000 writes:

    > The patch below adds support for custom HTTP headers in
    > download.file() and url().

    > My main motivation for this is performing basic http
    > authentication.  Some web sites do not support embedding
    > the credentials into the URI itself, they only work if the
    > username and password are sent in the HTTP headers. In
    > fact specifying the username and password in the URI has
    > been
    > deprecated.(https://en.wikipedia.org/wiki/Basic_access_authentication#URL_encoding)

    > Unfortunately this means that download.file() and url()
    > cannot access these password protected URLs. This patch fixes that.

Dear Gábor,
thank you very much for your valuable contribution!
R core members have been busy but now have reviewed your code
and liked it, and I have committed it to R-devel aka "the trunk"
(svn rev 75890).

So this should become part of the R x.y.0 release around April 2019.

Thanking you once more ...
merry Christmas and happy holidays !

Martin

    > I am happy to update the patch as needed.

    > Details:

   > * This patch adds supports for custom HTTP headers in download.file() and url().
   > * They both get a headers = NULL argument.
   > * This is implemented for the internal, wininet and libcurl methods.
   > * For other methods headers is silently ignored.
   > * For non-HTTP URLs headers is silently ignored.
   > * The headers argument must be a named character vector without NAs, or NULL.
   > * If headers is not named or it contains NAs, or the names contain
   > NAs, an error is thrown.
   > * For download.file() the method is chosen in R, and we a character
   > vector to C for libcurl and a collapsed string constant for internal
   > and wininet.
   > * For url() the method is only chosen in C, so we pass both a string
   > vector and the collapsed string vector to C. This is simpler than
   > collapsing in C.
   > * It is not possible to specify headers for file(), even though it handles URLs.
   > * The user agent (coming from the HTTPUserAgent options), will the the
   > first header, for the methods that need it together with the other
   > headers.
   > * We don't check for duplicate headers, just pass to the methods as
   > the user specified them.
   > * We test all methods.
   > * We have run the tests on macOS, Debian Linux and Windows 2016 Server.
   > 
   > You can also browse the changes here:
   > https://github.com/gaborcsardi/r-source/pull/3/files
   > You can also download the diff below from
   > https://github.com/gaborcsardi/r-source/pull/3.diff
   > 
   > Best,
   > Gabor

[....................]