[R] how to know if a file exists on a remote server?

[Baoqiang Cao]

I must say that your reminder is exactly what happened on me, that is,
the file is there but some of the downloaded files are corrupted. The
download.file didn't return anything alarming, but just could't open
some files. The problem was solved in my case by turning on
"method=wget".

Thanks again!

BC

On Tue, Nov 30, 2010 at 5:39 PM, steven mosher <moshersteven at gmail.com> wrote:
>  study  trycatch()
>
>  also, be awre that even with RCurl, that you may find the file there and
> then fail or lose
> the connection.
>
> worse still you may get a currupt file on download. So there is a lot of
> checking to do
> to make bullet proof code that downloads files.
>
>
>
>
>
> On Tue, Nov 30, 2010 at 3:16 PM, Baoqiang Cao <bqcaomail at gmail.com> wrote:
>>
>> Hi Georg,
>>
>> Your code does work, I mean, it doesn't give me any error message,
>> which is critical for me because I need use it in a loop and plus I
>> don't know how to catch error message. Before your message, I was
>> using download.file but the loop was stopped because of the error
>> message when a file doesn't exist. So I guess, the option
>> "method=wget" made the difference.
>>
>> To summarize (in case it is useful to others), there are (at least)
>> two ways to download files:
>>
>> 1) Georg Ruß:
>>  v = download.file(url,destf,method="wget")
>> if(v!=0) {
>> #download.file failed
>> }
>> #no error message though
>>
>> 2)
>>
>> Henrique Dallazuanna and Steven Mosher both suggested using RCurl,
>> here is an example code from Henrique for checking if a file exists on
>> a server:
>> "
>> library(RCurl)
>> h = basicHeaderGatherer()
>> Lines <- getURI("http://www.pdb.org/pdb/files/2J0S.1001",
>> headerfunction = h$update)
>> h$value()[['status']]
>>
>> If the status is 404, then not found. If exists then status should be 200.
>> "
>>
>> What a productive day!
>>
>> BC
>> On Tue, Nov 30, 2010 at 1:34 PM, Georg Ruß <research at georgruss.de> wrote:
>> > On 30/11/10 10:10:07, Baoqiang Cao wrote:
>> >> I'd like to download some data files from a remote server, the problem
>> >> here is that some of the files actually don't exist, which I don't
>> >> know before try. Just wondering if a function in R could tell me if a
>> >> file exists on a remote server?
>> >
>> > Hi Baoqiang,
>> >
>> > try downloading the file with R's download.file() function. Then you
>> > should examine the returned value.
>> >
>> > Citing a part of ?download.file below:
>> >
>> >>> Value:
>> >>> An (invisible) integer code, ‘0’ for success and non-zero for
>> >>> failure.  For the ‘"wget"’ and ‘"lynx"’ methods this is the status
>> >>> code returned by the external program.  The ‘"internal"’ method can
>> >>> return ‘1’, but will in most cases throw an error.
>> >
>> > So if you call your download via
>> >
>> > v <- download.file(url, destfile, method="wget")
>> >
>> > and v is not equal to zero, then the file is likely to be non-existent
>> > (at
>> > least the download failed). Note: the method "internal" doesn't really
>> > change the value of v, I just tried that. With "wget" it returns "0" for
>> > success and "2048" (or some other value) for non-success.
>> >
>> > Regards,
>> > Georg.
>> > --
>> > Research Assistant
>> > Otto-von-Guericke-Universität Magdeburg
>> > research at georgruss.de
>> > http://research.georgruss.de
>> >
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>