[R-sig-Geo] readGDAL does not close connections

[Roger Bivand]

On Thu, 6 Mar 2008, Paul Hiemstra wrote:

> Dear r-sig-geo list,
>
> I'm struggeling with readGDAL (rgdal package) not closing its file
> connections (I suspect). I am trying to construct a timeseries of
> rainfall using the following loop (conceptual code):
>
> pt = SpatialPoint()
> rainfall = numeric(length = length(listofiles))
> for(i in 1:length(listofiles)) {
>    grid = readGDAL(listofiles[i])
>    rainfall[i] = grd$value[overlay(grid, pt)]
> }
>
> There are more than 35000 grids (HDF5) that I loop over (2007 - 15
> minutes interval). After reading about 1000 grids my program fails:
>
> HDF5:"radar/2007/01/11/RAD_NL21_PCP_NA_200701111515.h5"://image1/image_data
> has GDAL driver HDF5Image
> and has 256 rows and 256 columns
> Error in gzfile(file, "r") : unable to open connection
> In addition: Warning message:
> In gzfile(file, "r") :
>  cannot open compressed file '/home/hiemstra/lib/R/rgdal/DESCRIPTION'

After an error like this, please also report what traceback() says. A 
connection is not an open file. I have just run a loop of 10000 readGDAL() 
without any connections left open - showConnections() shows open 
connections - and indeed as far as I know, readGDAL() does not itself use 
any connections mechanisms.

In your case, something is trying to read:

/home/hiemstra/lib/R/rgdal/DESCRIPTION

using a connections mechanism, but without traceback(), we don't know 
what.

Roger

>
> I suspect that readGDAL opens connections to the files, but does not
> close them and the program fails because too many files have been
> openened simultaneously. I do not get messages like "Closing GDAL
> dataset handle 0x8ff7900...  destroyed ... done." when I run the script.
> Does anybody on the r-sig-geo list have similar experiences?
>
> kind regards,
> Paul
>
> ps Additional information on my session
> > sessionInfo()
> R version 2.6.1 (2007-11-26)
> i486-pc-linux-gnu
>
> locale:
> LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> other attached packages:
> [1] rgdal_0.5-24   maptools_0.7-7 foreign_0.8-23 automap_0.4-3
> classInt_0.1-9
> [6] e1071_1.5-17   class_7.2-41   gstat_0.9-43   sp_0.9-23
>
> loaded via a namespace (and not attached):
> [1] grid_2.6.1      lattice_0.17-6  rcompgen_0.1-17
>
> I run Debian Lenny
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no