[R-sig-Geo] reading .hdf files
Jonathan Greenberg
jgrn at illinois.edu
Fri Mar 28 20:34:15 CET 2014
David:
I don't see any information in that file re: pixel size -- the corner
coordinates appear to be pixel (not geography) coordinates, so they
won't help you. If they did contain the right info, you could use
some basic text parsing to extract it from the output of gdalinfo().
So, you are left with a couple of choices:
1) Get the metadata on the file you are using from NOAA and see if
they can help you figure out the corner coordinates/pixel sizes OR;
2) (my suggestion) use a different data set. Why are you using this
particular dataset? If you want large-scale, well-calibrated NDVI,
the MODIS products are standardized and go back to 2000
(https://lpdaac.usgs.gov/products/modis_products_table/mod13q1).
Alternatively, if you want finer-scale NDVI (30m), you can get the
LANDSAT LEDAPS products (you want the surface reflectance product)
from Earth Explorer (http://landsat.usgs.gov/CDR_LSR.php)-- these go
back to at least 1985. From these, you can just calculate the
per-image NDVI yourself (it is a simple calculation: (NIR - RED) /
(NIR + RED) ).
--j
On Fri, Mar 28, 2014 at 2:07 PM, David Bradley <dododave at gmail.com> wrote:
> Hi Jonathan,
>
> I am trying to extract mean ndvi values for the wintering range of
> individual tree swallows. we have placed geolocators (light sensitive
> tracking devices) on birds, and I would like to study their migration
> patterns and timing of arrival/departure from non-breeding and stopover
> sites, and see if they time arrival at these sites to vegetation greening
> (as a measure of flying insect abundance).
>
> Thanks,
> David
>
>
>
> On 28 March 2014 15:02, David Bradley <dododave at gmail.com> wrote:
>>
>> Great!
>>
>> > gdalUtils:::gdalinfo("GVIX_NN_G16_C07_NVI_Y2011_P01.hdf")
>> [1] "Driver: HDF4Image/HDF4 Dataset"
>> [2] "Files: GVIX_NN_G16_C07_NVI_Y2011_P01.hdf"
>> [3] "Size is 2500, 904"
>> [4] "Coordinate System is `'"
>> [5] "Metadata:"
>> [6] " END_LATITUDE_RANGE=-55.15200043"
>> [7] " END_LONGITUDE_RANGE=180"
>> [8] " GRID_COLUMNS=2500"
>> [9] " GRID_ROWS=904"
>> [10] " MISSING=-999"
>> [11] " PROJECTION=Plate_Carree"
>> [12] " RANGE_MAX=1"
>> [13] " RANGE_MIN=-0.1000000015"
>> [14] " SCALED=1"
>> [15] " SCALED_MAX=1000"
>> [16] " SCALED_MIN=-100"
>> [17] " SCALED_MISSING=-999"
>> [18] " START_LATITUDE_RANGE=75.02400208"
>> [19] " START_LONGITUDE_RANGE=-180"
>> [20] " UNITS=NONE"
>> [21] "Corner Coordinates:"
>> [22] "Upper Left ( 0.0, 0.0)"
>> [23] "Lower Left ( 0.0, 904.0)"
>> [24] "Upper Right ( 2500.0, 0.0)"
>> [25] "Lower Right ( 2500.0, 904.0)"
>> [26] "Center ( 1250.0, 452.0)"
>> [27] "Band 1 Block=2500x400 Type=Int16, ColorInterp=Gray"
>>
>> If there any way this can be saved and re-applied to the tiff files and
>> the polygon shapefile?
>>
>>
>> On 28 March 2014 14:58, Jonathan Greenberg <jgrn at illinois.edu> wrote:
>>>
>>> It's not just the projection -- you need to know the upper left
>>> coordinates (or the full bounding box) and the pixel resolution, and
>>> set those as well. When you said you downloaded this from NOAA --
>>> what is the main site describing the dataset? What are you trying to
>>> do with this data?
>>>
>>> gdalUtils:::gdalinfo() will give you a "fuller" set of metadata of the
>>> input data than rgdal:::GDALinfo does, just as a note.
>>>
>>> --j
>>>
>>> On Fri, Mar 28, 2014 at 1:51 PM, dododave <dododave at gmail.com> wrote:
>>> > Yes, I checked the projection info from the newly converted tiff file,
>>> > and it
>>> > is blank. I even tried to set the projection with the command below to
>>> > match
>>> > my poly file;
>>> >
>>> > projection(rast) <- "+proj=longlat +datum=WGS84 +ellps=WGS84
>>> > +towgs84=0,0,0"
>>> >
>>> > but this isn't right. I tried to use the GDALinfo command to extract
>>> > the
>>> > metadata, but that doesn't work either. The ftp site I downloaded the
>>> > .hdf
>>> > files from is run by the NOAA, and I found some info;
>>> >
>>> > Table 2 Description of File attributes (Meta data)
>>> > Meta data tag Content
>>> > GRID_ROWS Number of rows for data grid
>>> > GRID_COLUMNS Number of columns for data grid
>>> > START_LATITUDE_RANGE The latitude of the grid's northern edge
>>> > END_LATITUDE_RANGE The latitude of the grid's southern edge
>>> > START_LONGITUDE_RANGE The longitude of the grid's western edge
>>> > END_LONGITUDE_RANGE The longitude of the grid's eastern edge
>>> > PROJECTION Projection type
>>> >
>>> > But how do I view the metadata?
>>> >
>>> >
>>> >
>>> > Jonathan Greenberg-3 wrote
>>> >> I want to follow up with this -- I confirmed Barry's suspicion that
>>> >> that file you sent above ("GVIX_NL_G16_C07_NVI_Y2005_P26.hdf") has no
>>> >> projection information (or bounding box or pixel resolution), so there
>>> >> is no way to do the extract unless that is fixed. Run a:
>>> >>
>>> >> gdalinfo("GVIX_NL_G16_C07_NVI_Y2005_P26.hdf")
>>> >>
>>> >> and you'll see there is not a properly set projection -- the .tif,
>>> >> thus, also doesn't have any projection info. This is an issue with
>>> >> the input file, not with any of our functions. You will need to set
>>> >> your own projection, bounding box, and pixel sizes on the raster via
>>> >> e.g. ?projection, ?res, ?extent
>>> >>
>>> >> What are these files? Are they satellite data? Do you know if there
>>> >> is another source for them that may have the information set
>>> >> "properly"?
>>> >>
>>> >> --j
>>> >>
>>> >> On Fri, Mar 28, 2014 at 11:38 AM, Barry Rowlingson
>>> >> <
>>> >
>>> >> b.rowlingson at .ac
>>> >
>>> >> > wrote:
>>> >>> On Fri, Mar 28, 2014 at 4:11 PM, dododave <
>>> >
>>> >> dododave@
>>> >
>>> >> > wrote:
>>> >>>
>>> >>>> I am now able to convert the hdf files into GeoTiffs. I have a
>>> >>>> follow-up
>>> >>>> question though.
>>> >>>>
>>> >>>> I now need to extract the mean ndvi value from within a polygon
>>> >>>> overlaid
>>> >>>> on
>>> >>>> that GeoTiff. The issue I am having is that the code is returning
>>> >>>> NULL
>>> >>>> when
>>> >>>> I do.
>>> >>>>
>>> >>>> > rast <- raster("2011_P01.tif")
>>> >>>> > mean.ndvi <- extract(rast, poly, fun=mean)
>>> >>>> > mean.ndvi
>>> >>>> [[1]]
>>> >>>> NULL
>>> >>>>
>>> >>>>
>>> >>>> Here is the output when I call rast;
>>> >>>> > rast
>>> >>>> class : RasterLayer
>>> >>>> dimensions : 904, 2500, 2260000 (nrow, ncol, ncell)
>>> >>>> resolution : 1, 1 (x, y)
>>> >>>> extent : 0, 2500, 0, 904 (xmin, xmax, ymin, ymax)
>>> >>>> coord. ref. : NA
>>> >>>> data source : /Volumes/GOLONDRINA/tiff/2011_P01.tif
>>> >>>> names : X2011_P01
>>> >>>> values : -32768, 32767 (min, max)
>>> >>>>
>>> >>>> Help appreciated!
>>> >>>>
>>> >>>
>>> >>> Are you sure your polygon is in the right place?
>>> >>>
>>> >>> r=raster()
>>> >>> r[]=runif(180*360)
>>> >>> pts=data.frame(x=c(-50,50,50,-50,-50),y=c(-50,-50,50,50,-50))
>>> >>> poly = SpatialPolygons(list(Polygons(list(Polygon(pts)),ID=1)))
>>> >>> extract(r,poly,fun=mean)
>>> >>> [1] 0.4997085
>>> >>>
>>> >>> - so that;s all good, but if I move the polygon way off the raster:
>>> >>>
>>> >>> poly2 = SpatialPolygons(list(Polygons(list(Polygon(pts+200)),ID=1)))
>>> >>> extract(r,poly2,fun=mean)
>>> >>> [[1]]
>>> >>> NULL
>>> >>>
>>> >>> I get what you're getting. Check your polygon coordinates and CRS
>>> >>> string
>>> >>> is correct...
>>> >>>
>>> >>> Barry
>>> >>>
>>> >>> [[alternative HTML version deleted]]
>>> >>>
>>> >>> _______________________________________________
>>> >>> R-sig-Geo mailing list
>>> >>>
>>> >
>>> >> R-sig-Geo@
>>> >
>>> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Jonathan A. Greenberg, PhD
>>> >> Assistant Professor
>>> >> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>>> >> Department of Geography and Geographic Information Science
>>> >> University of Illinois at Urbana-Champaign
>>> >> 259 Computing Applications Building, MC-150
>>> >> 605 East Springfield Avenue
>>> >> Champaign, IL 61820-6371
>>> >> Phone: 217-300-1924
>>> >> http://www.geog.illinois.edu/~jgrn/
>>> >> AIM: jgrn307, MSN:
>>> >
>>> >> jgrn307@
>>> >
>>> >> , Gchat: jgrn307, Skype: jgrn3007
>>> >>
>>> >> _______________________________________________
>>> >> R-sig-Geo mailing list
>>> >
>>> >> R-sig-Geo@
>>> >
>>> >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > View this message in context:
>>> > http://r-sig-geo.2731867.n2.nabble.com/reading-hdf-files-tp7586038p7586077.html
>>> > Sent from the R-sig-geo mailing list archive at Nabble.com.
>>> >
>>> > _______________________________________________
>>> > R-sig-Geo mailing list
>>> > R-sig-Geo at r-project.org
>>> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>>
>>>
>>> --
>>> Jonathan A. Greenberg, PhD
>>> Assistant Professor
>>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>>> Department of Geography and Geographic Information Science
>>> University of Illinois at Urbana-Champaign
>>> 259 Computing Applications Building, MC-150
>>> 605 East Springfield Avenue
>>> Champaign, IL 61820-6371
>>> Phone: 217-300-1924
>>> http://www.geog.illinois.edu/~jgrn/
>>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>>
>>
>>
>>
>> --
>> David Bradley
>> A346 Paisley Road, Guelph
>> Ontario, N1H 2P9
>
>
>
>
> --
> David Bradley
> A346 Paisley Road, Guelph
> Ontario, N1H 2P9
--
Jonathan A. Greenberg, PhD
Assistant Professor
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
259 Computing Applications Building, MC-150
605 East Springfield Avenue
Champaign, IL 61820-6371
Phone: 217-300-1924
http://www.geog.illinois.edu/~jgrn/
AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
More information about the R-sig-Geo
mailing list