[R-pkg-devel] download.file and https

[Paul Gilbert]

On 07/02/2015 11:52 PM, Duncan Murdoch wrote:
> On 03/07/2015 5:24 AM, Paul Gilbert wrote:
>>
>>
>> On 07/02/2015 10:52 PM, Henrik Bengtsson wrote:
>>>   From R 3.2.0, check:
>>>
>>>> capabilities("libcurl")
>>> libcurl
>>>      TRUE
>>>
>>> TRUE means R was built such that HTTPS is supported.  If you see
>>> FALSE, make sure libcurl is available when/if you build R from source.
>>
>> I do have TRUE for this. The default behaviour still does not work.
>
> Have you tried specifying the protocol explicitly, i.e.
>
>   download.file("https://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv", destfile = tmp, method="libcurl")
>
> ?

Yes. The default does not accept a string with https.  It gives 
"unsupported URL scheme".

A few people have mentioned workarounds or setting options. But, as 
Joshua mentioned, the question is really about best practice to make a 
package robust on all platforms, and tests that pass on CRAN.

Paul

>
> Duncan Murdoch
>
>>
>> Paul
>>
>>>
>>> /Henrik
>>>
>>> On Thu, Jul 2, 2015 at 7:46 PM, Paul Gilbert <pgilbert902 at gmail.com> wrote:
>>>> (This problem with download.file() affects quantmod, and possibly several
>>>> other packages. e.g. getSymbols('M2',src='FRED') fails.)
>>>>
>>>> I think the St Louis Fed has moved to using https for connections, and I
>>>> believe all the US government web sites are doing this. An http request is
>>>> automatically switched to https. The default download.file method does not
>>>> seem to handle this, but method="wget" does:
>>>>
>>>>>    tmp <- tempfile()
>>>>>
>>>>> download.file("http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv",
>>>>> destfile = tmp)
>>>> trying URL
>>>> 'http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv'
>>>> Error in
>>>> download.file("http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv",
>>>> :
>>>>     cannot open URL
>>>> 'http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv'
>>>>>
>>>>>
>>>>>
>>>>> download.file("http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv",
>>>>> destfile = tmp, method="wget")
>>>> --2015-07-02 22:29:49--
>>>> http://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv
>>>> Resolving research.stlouisfed.org (research.stlouisfed.org)... 65.89.18.120
>>>> Connecting to research.stlouisfed.org
>>>> (research.stlouisfed.org)|65.89.18.120|:80... connected.
>>>> HTTP request sent, awaiting response... 301 Moved Permanently
>>>> Location:
>>>> https://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv
>>>> [following]
>>>> --2015-07-02 22:29:49--
>>>> https://research.stlouisfed.org/fred2/series/M2/downloaddata/M2.csv
>>>> Connecting to research.stlouisfed.org
>>>> (research.stlouisfed.org)|65.89.18.120|:443... connected.
>>>> HTTP request sent, awaiting response... 200 OK
>>>> Length: unspecified [text/x-comma-separated-values]
>>>> Saving to: ‘/tmp/RtmpOX7kA1/file1ba639d7fd0f’
>>>>
>>>>       [ <=>                                   ] 34,519       178KB/s   in 0.2s
>>>>
>>>> 2015-07-02 22:29:50 (178 KB/s) - ‘/tmp/RtmpOX7kA1/file1ba639d7fd0f’ saved
>>>> [34519]
>>>>
>>>> Paul
>>>>
>>>> ______________________________________________
>>>> R-package-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>> ______________________________________________
>> R-package-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>