[R] Unexpected values obtained when reading in data using ncdf and ncdf4

[Louise Mair]

Dear R Users,

I am encountering a problem when reading nc files into R using the ncdf and ncdf4 libraries. The nc files are too large to attach an example (but if someone is interested in helping out I could send a file privately via an online drive), but the code is basic:

for(i in 1:length(thesenames[,1])){
   data <- nc_open(paste(INDIR, thesenames[i,c("wholename")], sep=""), write=F)
   d.vars <- names(data$var)
   d.size <- (data$var[[length(d.vars)]])$size

   # Obtaining longitude and latitude values
   d.lon <- as.vector(ncvar_get(data, varid="lon", start=c(1,1), count=c(d.size[1],d.size[2])))
   d.lat <- as.vector(ncvar_get(data, varid="lat", start=c(1,1), count=c(d.size[1],d.size[2])))

   # Obtaining climate data values
   df.clim <- data.frame(rn=seq(1:length(d.lon)))
   for(y in 1:d.size[3]){
     df.clim[,1+y] <- as.vector(ncvar_get(data, varid=d.vars[length(d.vars)], start=c(1,1,y), count=c(d.size[1],d.size[2],1)))
      names(df.clim)[1+y] <- paste("y",y,sep="")  }
   tosummarise[,,i] <- as.matrix(df.clim[,-1])
}

The data are temperature or precipitation, across space and time.

For most of the >250 files I have, there are no problems, but for around 8 of these files, I get strange values. The data should be within a relatively narrow range, yet I get values such as -8.246508e+07  or  7.659506e+11. The particularly strange part is that these kind of values occur at regularly spaced intervals across the data, usually within a single time step.

I have the same problem (including the exact same strange values) when using ArcMap, yet the data provider assures me that the data look normal when using CDO (climate data operators) to view them, and that there are no missing values.

I realise this is very difficult to diagnose without the nc files themselves, so my questions are (1) Has anyone encountered something like this before?, (2) Is there something I am failing to specify in the code when reading in?, and (3) Is anyone interested in digging into this and willing to play around with the nc files if I make them available privately?

Thanks very much in advance!
Louise





	[[alternative HTML version deleted]]