[R-sig-Geo] raster pkg: overwriting the dimensions detected by netcdf4 (PERSIANN-CDR)

Wed Feb 3 17:38:42 CET 2016

On 3 February 2016 at 12:56, Michael Sumner <mdsumner at gmail.com> wrote:
>
>
> On Thu, 4 Feb 2016 at 02:20 MAURICIO ZAMBRANO BIGIARINI
> <mauricio.zambrano at ufrontera.cl> wrote:
>>
>> On 3 February 2016 at 11:34, Michael Sumner <mdsumner at gmail.com> wrote:
>> >
>> >
>> > On Thu, 4 Feb 2016 at 01:26 MAURICIO ZAMBRANO BIGIARINI
>> > <mauricio.zambrano at ufrontera.cl> wrote:
>> >>
>> >> Dear spatial community,
>> >>
>> >> While reading some netCDFfiles with the raster package , I got a
>> >> "rotated" file, where the columns where read as rows and vice versa:
>> >>
>> >> ------------- START ----------------
>> >> x <- raster("PERSIANN-CDR_v01r01_20090101_c20140523.nc")
>> >> Loading required namespace: ncdf4
>> >> plot(x)
>> >> x
>> >> class       : RasterLayer
>> >> dimensions  : 1440, 480, 691200  (nrow, ncol, ncell)
>> >> resolution  : 0.25, 0.25  (x, y)
>> >> extent      : -60, 60, 0, 360  (xmin, xmax, ymin, ymax)
>> >> coord. ref. : NA
>> >> data source : /home/hzambran/PERSIANN-CDR_v01r01_20090101_c20140523.nc
>> >> names       :
>> >> NOAA.Climate.Data.Record.of.PERSIANN.CDR.daily.precipitation
>> >> z-value     : 2009-01-01
>> >> zvar        : precipitation
>> >> ------------- END ----------------
>> >>
>> >
>> > Transpose might be sufficient?
>> >
>> > tx <- t(x)
>> >
>> > plot(tx)
>> > tx
>>
>>
>> Thanks Michale,
>>
>> In fact, that is what I'm doing for reading the files, and the spatial
>> extent is correct and the precipitation values also seems to be right.
>>
>> However, my question was more "conceptual", in the sense that the
>> provider says that the original file is NOT rotated, while the only
>> way  I have to get the correct files is rotating the original file
>> with the 't' command you mentioned.
>>
>
>> In addition, while reading the file with the latest version of GRASS
>> GIS (7.0.3), and I got the following error message:
>>
>> "
>> (Tue Feb  2 19:41:29 2016)
>> r.in.gdal input=/home/hzambran/PERSIANN-CDR_v01r01_20090101_c20140523.nc
>> output=PERSIANN_CDR_v01r01_20090101_c20140523 -e
>> Warning 1: dimension #2 (lat) is not a Longitude/X
>> dimension.
>> Warning 1: dimension #1 (lon) is not a Latitude/Y dimension.
>> ERROR: Input raster map is flipped or rotated - cannot import. You may
>> use 'gdalwarp' to transform the map to North-up.
>> "
>>
>>
>> So, I'm quite sure that the original file IS rotated, but I didn't
>> have more technical arguments to give to the data provider to
>> demonstrate that the file is rotated, because they insist that the
>> rotation is because R is not reading the file in the proper way....
>>
>
>
> I wouldn't bother with the conversation if I were you. Just get your toolkit
> in order and make sure you know what it's doing and why.  If you explore
> their tools you might quickly understand why they're so adamant, but it's
> not worth the time IMO.

>
> library(fortunes)
> fortune("illogical")
>
> If there's a predictable pattern in the file you might explore a patch for
> raster to auto-detect and handle it? I'd be happy to do that - if you
> provide the data and as much information about it and its kind as you can
> find.  If you want to know more I would start with ncdf4 directly, so you
> see what raster has to do down there.

Thanks your very much Michael and Chris for your comments.

In fact, reading the file with gdal_translate I got a warning message
about the dimensions:

Checking gdal_installation...
Scanning for GDAL installations...
Checking Sys.which...
GDAL version 1.11.2
GDAL command being used: "/usr/bin/gdal_translate" -of "GTiff" -b "1"
"NETCDF:PERSIANN-CDR_v01r01_20090101_c20140523.nc:precipitation"
"PERSIANN-CDR_v01r01_20090101_c20140523.nc.tif"
Warning 1: dimension #2 (lat) is not a Longitude/X dimension.
Warning 1: dimension #1 (lon) is not a Latitude/Y dimension.

also, 'ncview' indicates that the Y variable corresponds to longitude,
going from 0.125 to 359.875, while the X variable corresponds to
latitude, with values going from 59.75 to -59.875.

However, 'ncdump -h ' shows that the attributes are correctly stored
within the file (but with no extent):

netcdf PERSIANN-CDR_v01r01_20090101_c20140523 {
dimensions:
time = 1 ;
lat = 480 ;
lon = 1440 ;
nv = 2 ;
variables:
int time(time) ;
time:standard_name = "time" ;
time:long_name = "time" ;
time:units = "days since 1979-01-01 0:0:0" ;
float lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:valid_max = 60.f ;
lat:valid_min = -60.f ;
lat:bounds = "lat_bnds" ;
float lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:valid_max = 360.f ;
lon:valid_min = 0.f ;
lon:bounds = "lon_bnds" ;

>
> BTW, there's nothing strange about the way raster handles this in general,
> if you peek under the hood in many software packages they all have the same
> zoo of exceptions and special cases. NetCDF is incredibly general and that
> flexibility gets pretty thoroughly abused. I have no patience for talk of
> standards and conventions when it comes to this format, the data that floods
> in just has to be dealt with, and that means your toolkit is full of special
> cases (raster the details exceptionally well for its portion of file
> variants).

Thanks for that comment (and the fortunes citation). In the end, I'll
continue rotating the file with 't' knowing that the contnet of the
file is actually rotated.

Kind regards,

Mauricio

=====================================
Dept. of Civil Engineering
Faculty of Engineering and Sciences
Universidad de La Frontera
PO Box 54-D, Temuco, Chile
http://ingenieriacivil.ufro.cl/
=====================================
mailto     : mauricio.zambrano at ufrontera.cl
work-phone : +56 45 259 2812
=====================================
"In the end, it's not the years in your life that count.
 It's the life in your years". (Abraham Lincoln)
=====================================
Linux user #454569 -- Linux Mint user

-- 
La información contenida en este correo electrónico y cualquier anexo o 
respuesta relacionada, puede contener datos e información confidencial y no 
puede ser usada o difundida por personas distintas a su(s) destinatario(s). 
Si usted no es el destinatario de esta comunicación, le informamos que 
cualquier divulgación, distribución o copia de esta información constituye 
un delito conforme a la ley chilena. Si lo ha recibido por error, por favor 
borre el mensaje y todos sus anexos y notifique al remitente.