[R] what does this err mean and how to solve it? Error in file(file, ifelse(append, "a", "w"))

[Prof Brian Ripley]

On Wed, 6 Oct 2010, Peter Langfelder wrote:

> you should close files that you do not use anymore. the maximum number
> of open files is likely 4000 or so. Use close(file) before you open
> the next one.

He/she said he did that, and as the limit on open connections is 128, 
that cannot be the sole cause.

Without even the 'at a mimimum information' requested in the posting 
guide let alone a reproducible example it is not possible to know the 
cause. But most OSes have utilities available to trace the 
opening/closing of files.  (We weren't told but this looks like 
Windows -- there I have used the monitors from sysinternals, now part 
of Microsoft.)

And BTW, the subject line omitted the associated warning which gives 
the underlying OS-level error -- that would have been a much better 
choice of subject.

>
> Peter
>
> On Wed, Oct 6, 2010 at 4:55 PM, Yong Wang <wangyong1 at gmail.com> wrote:
>> Dear List
>> I am running a loop downloading  web pages and save the html to a
>> temporary file (use download.file() )
>>  then read (using readLines)  it in for processing;
>> finally write useful info from each processed page to a unique file
>>
>> the problem is once the loop runs up to somewhere near  5000, it will
>> throw out an err like below and won't go further.
>>
>> ----------------------------------------------------------------
>> Error in file(file, ifelse(append, "a", "w")) :
>>  cannot open the connection
>> -----------------------------------------------------------------
>>
>> In the meantime, a request for new connection won't be successful, for
>> example, a request for the help page of "file" will trigger err below
>>
>> -----------------------------------------------------------------------
>>  ?file
>> Error in gzfile(file, "rb") : cannot open the connection
>> In addition: Warning message:
>> In gzfile(file, "rb") :
>>  cannot open compressed file
>> 'C:/PROGRA~1/R/R-211~1.1/library/stats/help/aliases.rds', probable
>> reason 'Too many open files'
>> -----------------------------------------------------------------------
>>
>> I am not sure if the problem is too many connections not closed. since
>> I close the file connection after each readLines.
>> checking with showConnections(all=T) does not show excessive
>> connections and closeAllConnections() does not help.
>>
>> Can any one help me on this?
>>
>>
>> Any answer highly appreciated.
>>
>> yong
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595