[R] remove a "corrupted file" after using download.file() with R on Windows 7

[Fabien Tarrade]

Hi Henrik,

> 1. It could be that a virus checker locks the file.
This is some internal document so I don't think so and this was monitor 
by our IT security team.
> 2. There are Windows software tools that identify which process locks
> a particular file, e.g. LockHunter (http://lockhunter.com/).  Those
> should help you figure out what's going on.
ok, thanks.
> 3. R.utils::downloadFile() tries it's best to download files
> atomically, i.e. it either gives you a fully downloaded file or not
> all.  In your case, you might still end up with a temporary corrupt
> file, but at least it will have a filename that is different than the
> one you ask for.
My issue is that if I run over 300'000 documents and I will get the same 
amount of corrupted files.
I will try some solution with R or try with python

Thanks
Cheers
Fabien
>
>> Hi there,
>>
>> Sometime download.file() failed to download the file and I would like to
>> remove the correspond file.
>> The issue is that I am not able to do it and Windows complain that the file
>> is use by another application.
>> I try to closeAllConnections(), or unlink() before removing the file but
>> without sucess.
>>
>> Any idea how I should proceed &
>>
>> Please find the code below
>>
>>   # consider warning as an error
>>    options(warn=2)
>>
>>    # try to download the file
>>    tryCatch({
>>      download.file(url,path_file,mode="wb",quiet=quiet)
>>      return(0)
>>    },error = function(e){
>>      if(verbose){
>>        print(e)
>>        print(e$message)
>>      }
>>      # close file when it failed
>>      if (file.exists(path_file)){
>>        closeAllConnections()
>>        #unlink(path_file, recursive=TRUE)
>>        #file.create(path_file,overwrite=TRUE,showWarning=TRUE)
>>        #system(paste0('open "', path_file, '"'))
>>        file.remove(path_file,overwrite=TRUE,showWarning=TRUE)
>>      }
>>      return(1)
>>      }
>> )
>>
>> Thanks a lot
>> Cheers
>> Fabien
>>
>> --
>> Dr Fabien Tarrade
>>
>> Quantitative Analyst/Developer - Data Scientist
>>
>> Senior data analyst specialised in the modelling, processing and statistical
>> treatment of data.
>> PhD in Physics, 10 years of experience as researcher at the forefront of
>> international scientific research.
>> Fascinated by finance and data modelling.
>>
>> Geneva, Switzerland
>>
>> Email : contact at fabien-tarrade.eu <mailto:contact at fabien-tarrade.eu>
>> Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>
>> Phone : +33 (0)6 14 78 70 90
>>
>> LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter
>> <https://twitter.com/fabtar> Google
>> <https://plus.google.com/+FabienTarradeProfile/posts> Facebook
>> <https://www.facebook.com/fabien.tarrade.eu> Google <skype:fabtarhiggs?call>
>> Xing <https://www.xing.com/profile/Fabien_Tarrade>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.

-- 
Dr Fabien Tarrade

Quantitative Analyst/Developer - Data Scientist

Senior data analyst specialised in the modelling, processing and 
statistical treatment of data.
PhD in Physics, 10 years of experience as researcher at the forefront of 
international scientific research.
Fascinated by finance and data modelling.

Geneva, Switzerland

Email : contact at fabien-tarrade.eu <mailto:contact at fabien-tarrade.eu>
Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>
Phone : +33 (0)6 14 78 70 90

LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter 
<https://twitter.com/fabtar> Google 
<https://plus.google.com/+FabienTarradeProfile/posts> Facebook 
<https://www.facebook.com/fabien.tarrade.eu> Google 
<skype:fabtarhiggs?call> Xing <https://www.xing.com/profile/Fabien_Tarrade>