[R-sig-Geo] Raster Data Management Advice

[Alexander Ilich]

Thank you everyone for the advice. I have some things to look into.

Thanks,
Alex
________________________________
From: Michael Sumner <mdsumner using gmail.com>
Sent: Sunday, October 9, 2022 5:01 AM
To: Alexander Ilich <ailich using mail.usf.edu>
Cc: r-sig-geo using r-project.org <r-sig-geo using r-project.org>
Subject: Re: [R-sig-Geo] Raster Data Management Advice

I would set up a polygon of the bounding box (in the native projection) of each raster source, and use fields on those polygons to store the details of interest: xmin,xmax,ymin,ymax, dimension, resolution, crs, and your other details. Then hone in a areas of interest for different tasks to see what set of overlapping data you have for it.

There's a lot of fanfare about STAC, but it's really just a JSON-format with some of the information you could store on a simple polygon dataset ...with STAC as with so many formats you'd have to shoehorn your data into that more restrictive form (you can always spit out STAC as a side product of your own rich summary for less sophisticated uses).

The crux is keeping the details of the source's native projection independent from the representation you use to query it spatially IMO, just record what's there. Further, the GDAL warper app-lib (one level below the gdalwarp.exe) is the right tool for doing general reads, of any number of sources into one specific window of your choosing in any projection (you could use your dataset described above to limit which sources get included). You can easily see what you'd get by merging any number of sources together, and of course more nuanced situations like a sensible background with more detailed layers merged over that is very valuable.

Cheers, Mike



On Sat, Oct 8, 2022 at 3:35 AM Alexander Ilich <ailich using mail.usf.edu<mailto:ailich using mail.usf.edu>> wrote:
Hi, I was wondering if anyone has some advice on how to organize raster
data so that it is easily queryable by various attributes (e.g. find me all
the rasters of data type bathymetry, collected by this organization with
10m resolution or finer ). Currently we have data on a server organized
often by when/where it was collected but that can make it difficult to find
specific rasters that meet a certain criteria. I've created a table as a
csv file on github <https://github.com/ailich/WFS_Multibeam_Metadata<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Failich%2FWFS_Multibeam_Metadata&data=05%7C01%7Cailich%40usf.edu%7Cc45fb8db339c4f5e12eb08daa9d4f41f%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C638009029357802723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YFHL%2B7tp5i2u6vFxTo5L3zgCu6auS2ttvmttPrNjR9M%3D&reserved=0>> where
each row is a raster and it has various column attributes describing it
(e.g. who collected it, what sonar was used, resolution, coordinate system,
etc) and a path to the filename as a temporary solution, but I think some
type of spatial database that would allow for querying and then reading
into R as terra objects, as well as into QGIS and ArcGIS as layers for
visualization would be optimal as multiple project members use these data.
Tools I've come across that seem potentially useful include PostGIS and
Geopackage, but I'm not entirely sure how to properly set them up or if
they'd suit my needs. Any advice would be greatly appreciated.

Thanks,
Alex

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo using r-project.org<mailto:R-sig-Geo using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-geo<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C01%7Cailich%40usf.edu%7Cc45fb8db339c4f5e12eb08daa9d4f41f%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C638009029357802723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=z4tucvFujxVkXKkVbv90Wx372fqYdP4VQ5R4qc3ERJ8%3D&reserved=0>


--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsumner using gmail.com<mailto:mdsumner using gmail.com>

[EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe.

	[[alternative HTML version deleted]]