[R] file reading problem unique to windows. Err info: Error in file(file, ifelse(append, "a", "w")). cannot open the connection

[Yong Wang]

Thanks a lot, Prof. Ripley. the problem must be  download.file() prior
R-patched 2.12.
for each loop, I tired a couple of candidate links and only one or
none will work, this must accumulate soon to an amount of unclosed
destination files beyond the tolerance of windows.
I updated R from 2.10 to R-patched 2.12 and the problem gone.


On Thu, Nov 25, 2010 at 3:09 AM, Prof Brian Ripley
<ripley at stats.ox.ac.uk> wrote:
> We don't have any of the information asked for in the posting guide, such as
> your version of R, reproducible example ....
>
> But please try R-patched, since this might be
>
>    • download.file() could leave the destination file open if the URL
>      was not able to be opened.  (PR#14414)
>
> (If you had followed the posting guide you would have tried R-patched before
> posting ....)
>
>
> On Wed, 24 Nov 2010, Yong Wang wrote:
>
>> Dear List
>>
>> I asked this question before, got some tips but can't get it solved.
>
> Where?  You didn't give a reference, and it would have helped the helpers.
>
>> it is clear now that this problem only occurs when run on windows (I
>> tested it on windows XP) nothing wrong at all when run on unix.
>> unfortunately, sometimes I have to run it on windows,
>> I appreciate any suggestion on how to circumvent this problem when run
>> on windows.
>> below is the problem description I submitted before.
>>
>> #############################################################
>>
>> I am running a loop downloading  web pages and save the html to a
>> temporary file (use download.file() )
>> then read (using readLines)  it in for processing;
>> finally write useful info from each processed page to a unique file
>>
>> the problem is once the loop runs up to somewhere near  5000, it will
>> throw out an err like below and won't go further.
>>
>> ----------------------------------------------------------------
>> Error in file(file, ifelse(append, "a", "w")) :
>> cannot open the connection
>> -----------------------------------------------------------------
>>
>> In the meantime, a request for new connection won't be successful, for
>> example, a request for the help page of "file" will trigger err below
>>
>> -----------------------------------------------------------------------
>> ?file
>> Error in gzfile(file, "rb") : cannot open the connection
>> In addition: Warning message:
>> In gzfile(file, "rb") :
>> cannot open compressed file
>> 'C:/PROGRA~1/R/R-211~1.1/library/stats/help/aliases.rds', probable
>> reason 'Too many open files'
>> -----------------------------------------------------------------------
>>
>> I am not sure if the problem is too many connections not closed. since
>> I close the file connection after each readLines.
>> checking with showConnections(all=T) does not show excessive
>> connections and closeAllConnections() does not help.
>>
>> Can any one help me on this?
>>
>>
>> Any answer highly appreciated.
>>
>> yong
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> --
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595