[R-sig-Geo] [RGDAL] Error with writeGDAL to pcraster maps (in a loop)

[Zack Holden]

Hi Phillipe,
I've had problems in the past reading many NetCDF files with having
too many open files. I don't know if this is related to the issue
you're seeing, but I'm attaching a link to to my solution for that
problem. You could have an upper limit set on your machine to the
number of files that can be opened.

http://r-sig-geo.2731867.n2.nabble.com/solved-raster-ncdf4-error-too-many-open-files-td7583723.html

Also, it might help others diagnose your problem if you tell us what
OS you are using and provide information about your R session.

I hope this helps,

Zack


On Tue, Apr 15, 2014 at 3:12 AM, Philippe R
<philippe.roudier at jrc.ec.europa.eu> wrote:
> Hello everyone
>
> My problem is the following : basically I need to pick up daily
> precipitation data from a netcdf file (50000 days) to create 50000 pcraster
> maps (one for each day).
> I am therefore doing a loop using RGDAL that looks like:
>
> /for (d in 1:50000){
>
> tget<-readGDAL("R:/mypath/target.map")   ##here I load my .map target file
>
> tget at data<-new_data[d]    #I change the data (replacing with the on efrom
> Netcdf)
>
> writeGDAL(tget, fname=name_out,
> drivername='PCRaster',type="Float32",mvFlag=-9999)   #export to .map files
> }
> /
> This works well until d=509 (and outputs are perfectly fine) and then it
> crashes, and I got the error message
>
> /Error in .local(.Object, ...) : Dataset copy failed
> Calls: writeGDAL ... saveDataset -> new -> initialize -> initialize ->
> .local -> .Call
> /
> The point is that if I start the loop at e.g. d=100, it crashes at 509+100.
> So, it does not seem to be a problem of syntax.
> So I suspect it may be a problem of memory (but I have enough disk space
> left)….maybe I should close the connection everytime but GDAL.close(tget)
> does not work
>
> /Error in slot(dataset, "handle") :
>   no slot of name "handle" for this object of class "SpatialGridDataFrame"
> /
> The very weird thing is that it happens when I run it on the cluster. If
> doing it locally (windows), I have no problem when d=509…
>
> Do you have any idea?
>
> Best,
>
>
>
>
> --
> View this message in context: http://r-sig-geo.2731867.n2.nabble.com/RGDAL-Error-with-writeGDAL-to-pcraster-maps-in-a-loop-tp7586213.html
> Sent from the R-sig-geo mailing list archive at Nabble.com.
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo