[Rd] Issues with libcurl + HTTP status codes (eg. 403, 404)

Tue Aug 25 22:33:39 CEST 2015

On 08/25/2015 01:30 PM, Kevin Ushey wrote:
> Hi Martin,
>
> Indeed it does (and I should have confirmed myself with R-patched and R-devel
> before posting...)

actually I don't know that it does -- it addresses the symptom but I think there 
should be an error from libcurl on the 403 / 404 rather than from read.dcf on 
error page...

Martin

>
> Thanks, and sorry for the noise.
> Kevin
>
>
> On Tue, Aug 25, 2015, 13:11 Martin Morgan <mtmorgan at fredhutch.org
> <mailto:mtmorgan at fredhutch.org>> wrote:
>
>     On 08/25/2015 12:54 PM, Kevin Ushey wrote:
>      > Hi all,
>      >
>      > The following fails for me (on OS X, although I imagine it's the same
>      > on other platforms using libcurl):
>      >
>      >      options(download.file.method = "libcurl")
>      >      options(repos = c(CRAN = "https://cran.rstudio.com/", CRANextra =
>      > "http://www.stats.ox.ac.uk/pub/RWin"))
>      >      install.packages("lattice") ## could be any package
>      >
>      > gives me:
>      >
>      >      > options(download.file.method = "libcurl")
>      >      > options(repos = c(CRAN = "https://cran.rstudio.com/", CRANextra
>      > = "http://www.stats.ox.ac.uk/pub/RWin"))
>      >      > install.packages("lattice") ## coudl be any package
>      >      Installing package into ‘/Users/kevinushey/Library/R/3.2/library’
>      >      (as ‘lib’ is unspecified)
>      >      Error: Line starting '<!DOCTYPE HTML PUBLI ...' is malformed!
>      >
>      > This seems to come from a call to `available.packages()` to a URL that
>      > doesn't exist on the server (likely when querying PACKAGES on the
>      > CRANextra repo)
>      >
>      > Eg.
>      >
>      >      > URL <- "http://www.stats.ox.ac.uk/pub/RWin"
>      >      > available.packages(URL, method = "internal")
>      >      Warning: unable to access index for repository
>      > http://www.stats.ox.ac.uk/pub/RWin
>      >           Package Version Priority Depends Imports LinkingTo Suggests
>      > Enhances License License_is_FOSS
>      >          License_restricts_use OS_type Archs MD5sum NeedsCompilation
>      > File Repository
>      >      > available.packages(URL, method = "libcurl")
>      >      Error: Line starting '<!DOCTYPE HTML PUBLI ...' is malformed!
>      >
>      > It looks like libcurl downloads and retrieves the 403 page itself,
>      > rather than reporting that it was actually forbidden, e.g.:
>      >
>      >      >
>     download.file("http://www.stats.ox.ac.uk/pub/RWin/bin/macosx/mavericks/contrib/3.2/PACKAGES.gz",
>      > tempfile(), method = "libcurl")
>      >      trying URL
>     'http://www.stats.ox.ac.uk/pub/RWin/bin/macosx/mavericks/contrib/3.2/PACKAGES.gz'
>      >      Content type 'text/html; charset=iso-8859-1' length 339 bytes
>      >      ==================================================
>      >      downloaded 339 bytes
>      >
>      > Using `method = "internal"` gives an error related to the inability to
>      > access that URL due to the HTTP status 403.
>      >
>      > The overarching issue here is that package installation shouldn't fail
>      > even if libcurl fails to access one of the repositories set.
>      >
>
>     With
>
>       > R.version.string
>     [1] "R version 3.2.2 Patched (2015-08-25 r69179)"
>
>     the behavior is to warn with an indication of the repository for which the
>     problem occurs
>
>       > URL <- "http://www.stats.ox.ac.uk/pub/RWin"
>       > available.packages(URL, method="libcurl")
>     Warning: unable to access index for repository
>     http://www.stats.ox.ac.uk/pub/RWin:
>         Line starting '<!DOCTYPE HTML PUBLI ...' is malformed!
>            Package Version Priority Depends Imports LinkingTo Suggests Enhances
>            License License_is_FOSS License_restricts_use OS_type Archs MD5sum
>            NeedsCompilation File Repository
>       > available.packages(URL, method="internal")
>     Warning: unable to access index for repository
>     http://www.stats.ox.ac.uk/pub/RWin:
>         cannot open URL 'http://www.stats.ox.ac.uk/pub/RWin/PACKAGES'
>            Package Version Priority Depends Imports LinkingTo Suggests Enhances
>            License License_is_FOSS License_restricts_use OS_type Archs MD5sum
>            NeedsCompilation File Repository
>
>     Does that work for you / address the problem?
>
>     Martin
>
>      >> sessionInfo()
>      > R version 3.2.2 (2015-08-14)
>      > Platform: x86_64-apple-darwin13.4.0 (64-bit)
>      > Running under: OS X 10.10.4 (Yosemite)
>      >
>      > locale:
>      > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8
>      >
>      > attached base packages:
>      > [1] stats     graphics  grDevices utils     datasets  methods   base
>      >
>      > other attached packages:
>      > [1] testthat_0.8.1.0.99  knitr_1.11           devtools_1.5.0.9001
>      > [4] BiocInstaller_1.15.5
>      >
>      > loaded via a namespace (and not attached):
>      >   [1] httr_1.0.0     R6_2.0.0.9000  tools_3.2.2    parallel_3.2.2
>     whisker_0.3-2
>      >   [6] RCurl_1.95-4.1 memoise_0.2.1  stringr_0.6.2  digest_0.6.4
>       evaluate_0.7.2
>      >
>      > Thanks,
>      > Kevin
>      >
>      > ______________________________________________
>      > R-devel at r-project.org <mailto:R-devel at r-project.org> mailing list
>      > https://stat.ethz.ch/mailman/listinfo/r-devel
>      >
>
>
>     --
>     Computational Biology / Fred Hutchinson Cancer Research Center
>     1100 Fairview Ave. N.
>     PO Box 19024 Seattle, WA 98109
>
>     Location: Arnold Building M1 B861
>     Phone: (206) 667-2793
>

-- 
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793