[R-sig-Geo] subdatasets in rgdal/raster

[Michael Sumner]

I have, and it's good but it's not what I am after here.

I really want access to HDF4 via tight-coupling. It does work well in
R when you build GDAL right, especially with raster over the top to
clean everything up. If you can compile those drivers into GDAL it
does beg the question why have a wrapper like gdalUtils. (My take on
this is that Windows-support stops at the CRAN binary, and again this
is why raster has its own NetCDF wrapper).

Ultimately the logic for these rogue files belongs in GDAL and
rgdal/raster should just reflect that: rgdal2 is essentially that, but
doesn't offer much more and it's not clear how to "rasterize"
everything again.

:)



On Thu, Oct 16, 2014 at 2:27 AM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
> Mike:
>
> Have you taken a look at gdalUtils?  This is a wrapper for all the
> GDAL binaries (I am still working on adding in all those new tools
> that were recently released with version 1.11.1, but it should be up
> to date as of v 1.10.1).  gdalinfo that is part of gdalUtils gives you
> the full info dump (more, if less clean, output than rgdal's gdalInfo
> will give you), which you could parse for the info you wanted.  I also
> have a custom function "get_subdatasets" which sounds a lot like what
> you are describing.
>
> Give it a shot!  You will need to install GDAL yourself first -- check
> the help for suggestions on which flavor of GDAL I recommend.
>
> --j
>
> On Wed, Oct 15, 2014 at 6:23 AM, Michael Sumner <mdsumner at gmail.com> wrote:
>> Thanks Roger, I'm still hoping to be able to code at this level one day.
>>
>> I have gone off half-cocked, confusing things. I can get SDS from
>> NetCDF and HDF files with GDALinfo() with no problems - I just had to
>> turn off returnScaleOffset, which I think has a confounding of nbands
>> and subdatasets - since a datasource with SDS has no bands, but I need
>> to explore this more to be sure.
>>
>> Cheers, Mike.
>>
>>
>>
>>
>>
>> On Wed, Oct 15, 2014 at 5:50 PM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
>>> On Wed, 15 Oct 2014, Michael Sumner wrote:
>>>
>>>> Hello,
>>>>
>>>> As far as I know there's nothing in rgdal to determine existing
>>>> subdatasets (SDS) in HDF (or NetCDF for that matter).
>>>>
>>>> In raster, raster(x) will read from the first candidate *variable*,
>>>> and list the others with a warning, which is analogous to an SDS but
>>>> of course it's always providing this via ncdf/ncdf4 rather than via
>>>> rgdal.
>>>>
>>>> Is it true there's no way to find out SDS from HDF or NetCDF in rgdal?
>>>
>>>
>>> Because these formats are poorly organised (an XML tree describing the
>>> internal structure would be easy to parse, but self-descriptive metadata
>>> isn't usual here), GDALinfo() does not provide much.
>>>
>>> If you start by identifying the incantation used by command line gdalinfo to
>>> report the SDS metadata, then look at its source code, it should be possible
>>> to make some progress. I don't use either if I can help it, and do not have
>>> the necessary trial set of sample files. You'd need to find a robust route
>>> covering a wide range of files seen in the wild to allow for oddities in the
>>> ways things are organised. It is unfortunate to have to branch code on the
>>> selected driver, but here it is hard to avoid. Once GDALinfo is reporting
>>> the metadata in a predictable way, the other functions can be given
>>> appropriate argument values.
>>>
>>> Also consider looking at this from the point of view of rgdal2 in addition
>>> to rgdal; it could possibly be "closer" there and looking at both may make
>>> it easier to spot:
>>>
>>> https://github.com/thk686/rgdal2
>>>
>>> Contributions welcome!
>>>
>>> Roger
>>>
>>>
>>>>
>>>> I've not really explored this before much since compiling HDF was
>>>> unavailable to me. gdalUtils fills a gap but only by running the
>>>> installed command line tools, which obviously can be avoided with the
>>>> right features in rgdal/raster. I personally have always run the
>>>> command line tools and then jumped back to R (if I could).
>>>>
>>>> (The vector analogy in rgdal to listing SDS would be ogrListLayers).
>>>>
>>>> This is a missing capability in the raster package, somewhat disguised
>>>> because of the independent support for NetCDF. I guess hardly anyone
>>>> is using HDF? That's a shame since these R tools really help.  It's a
>>>> pretty complicated story all round and I keep re/discovering
>>>> interesting corners. If anyone is writing anything significant about
>>>> all this I'd be keen to be involved.
>>>>
>>>> Cheers, Mike.
>>>>
>>>>
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, Norwegian School of Economics,
>>> Helleveien 30, N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; fax +47 55 95 91 00
>>> e-mail: Roger.Bivand at nhh.no
>>>
>>
>>
>>
>> --
>> Michael Sumner
>> Software and Database Engineer
>> Australian Antarctic Division
>> Hobart, Australia
>> e-mail: mdsumner at gmail.com
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
>
>
> --
> Jonathan A. Greenberg, PhD
> Assistant Professor
> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
> Department of Geography and Geographic Information Science
> University of Illinois at Urbana-Champaign
> 259 Computing Applications Building, MC-150
> 605 East Springfield Avenue
> Champaign, IL  61820-6371
> Phone: 217-300-1924
> http://www.geog.illinois.edu/~jgrn/
> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007



-- 
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsumner at gmail.com