Dear list:
I've found the solution to my previous post which I'm including here in
case others encounter this problem.

The default upper limit on the number of files that can be open at any time
for my Scientific Linux 6 OS install is 1024, and is specified in the
/opt/etc/security file. Once I changed the upper file limit, I was able to
read/write many NetCDF files without a problem.

I was very careful to open/close/remove each file during processing, so I'm
not sure why these files were still considered  'open', but in any case,
I'm able to do what I need to now.

Zack


On Thu, May 23, 2013 at 4:41 PM, Zack Holden <zaholdenfs@gmail.com> wrote:

> Dear list,
> I am extracting data from thousands of NetCDF files, using the raster
> library to read in each file as follows:
>
> nclist <- list.files(nc.dir)
> for(iFile in 1:length(nclist)) {
>   r <- raster(nclist[iFile], varname = "myvar")
>   # extract data here....
> }  # end loop.
>
> I reach the file handler limit at around 1000 and receive the following
> error message:
> error: too many open files.
>
> I then move to the ncdf4 library, using nc_open() to open each file,
> retrieve the data I need, and then use nc_close() when I'm finished.
> However, I see the same error message after approximately 1000 files.
>
> The file handle limit on this system is more than 6 million, so I'm unsure
> why I'm seeing this error.
> Can anyone advise me on why this is occurring, and how I can avoid this
> problem?
>
> thanks in advance for any suggestions.
>
>  R session info:
> R version 3.0 x86_64-redhat-linux-gnu
>
>
>
>

	[[alternative HTML version deleted]]

