[R-sig-Geo] Earthdata CMR access

[Ben Tupper]

What I have can search for S3 resources, but I happen to focus on OPeNDAP
instead.

As far as I can tell, the ncdf4 (and RNetCDF) packages don't provide
utilities for establishing credentialed opendap connections.  That is
handled by sort of a third party or some kind of wrapper.   Without
credentials I can search, but without credentials I cannot open a
connection to the opendap resources the search yields.

If I am understanding the error message shown at the bottom of the README
(see below), the server is receiving the request but for lack of
credentials is refusing access.  I'm stumped.

https://github.com/BigelowLab/cmr


On Wed, Apr 13, 2022 at 2:18 PM Joe Lee <hyoklee using hdfgroup.org> wrote:
>
> Yes, that's the hard part that I'm referring to.
>
> Once you pass that step, you may also want to look at another example in
MATLAB [1]
> to access PODAAC data on S3 using the temporary token.
>
> I'd really appreciate your effort if you can share a working R-code
project
> that can both search with CMR and then access NASA data on S3.
>
> Regards,
>
> [1]
https://www.hdfeos.org/zoo/PODAAC/ATM_SURFACE_TEMP_HUM_WIND_PRES_day_mean_ECCO_V4r4_nc_c.m
>
>
> -----Original Message-----
> From: Ben Tupper <btupper using bigelow.org>
> Sent: Wednesday, April 13, 2022 1:09 PM
> To: Joe Lee <hyoklee using hdfgroup.org>
> Cc: r-sig-geo using r-project.org
> Subject: Re: [R-sig-Geo] Earthdata CMR access
>
> Hi,
>
> I was planning to use the httr package or the newish httr2 package which
should be able to handle the transactions.  Once a resource is tracked down
via search, I plan to use the ncdf4 package to access.
>
> I can add headers to the http requests, so presumably that can include
tokens.
>
> But I am finding even the simple stuff like using the API to generate a
token, as shown here
https://wiki.earthdata.nasa.gov/display/EL/How+to+Generate+a+User+Token
> ,  to be challenging.  Thud!
>
> Ben
>
> httr: https://CRAN.R-project.org/package=httr
> httr2: https://CRAN.R-project.org/package=httr2
> ncdf4: https://CRAN.R-project.org/package=ncdf4
>
> On Wed, Apr 13, 2022 at 1:03 PM Joe Lee <hyoklee using hdfgroup.org> wrote:
> >
> > Hi,
> >
> > I'm also interested in the solution that you're seeking since I've
> > written a Python-equivalent CMR+OPeNDAP workflow a very long time ago
[1].
> >
> > It's both very tricky and difficult nowadays because NASA tries to
> > constrain AWS egress cost via Earthdata Login authentication.
> >
> > Does R provide a cURL-like library + a temporary token-based
authentication?
> >
> > [1]
> > https://earthdata.nasa.gov/collaborate/open-data-services-and-software
> > /api/opendap/gesdisc-airs-opendap-cmr
> >
> >
> > -----Original Message-----
> > From: R-sig-Geo <r-sig-geo-bounces using r-project.org> On Behalf Of Ben
> > Tupper
> > Sent: Wednesday, April 13, 2022 11:41 AM
> > To: r-sig-geo <r-sig-geo using r-project.org>
> > Subject: [R-sig-Geo] Earthdata CMR access
> >
> > Hello,
> >
> > I am interested in accessing physical oceanography data from PO.DAAC (
> > https://earthdata.nasa.gov/eosdis/daacs/podaac) from within R using
> > opendap.  NASA is moving its data offerings to the cloud, and it is
> > adopting Common Metadata Repositories ("CMR",
> > https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html) for
searching.  CMR searching is in lieu of searching THREDDS catalogs.
> >
> > I'm finding the on-ramp to using CMR tricker than I had expected given
the sales pitch behind CMR (or maybe the wheels have come off my car!)  I
am wondering if anyone in this community has waded into using R-based CMR
to access cloud-based geospatial data, and, if so, might be willing to
share some insights or provide guidance?
> >
> > Regards,
> > Ben
> >
> > --
> > Ben Tupper (he/him)
> > Bigelow Laboratory for Ocean Science
> > East Boothbay, Maine
> > http://www.bigelow.org/
> > https://eco.bigelow.org
> >
> >         [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
>
>
> --
> Ben Tupper (he/him)
> Bigelow Laboratory for Ocean Science
> East Boothbay, Maine
> http://www.bigelow.org/
> https://eco.bigelow.org



-- 
Ben Tupper (he/him)
Bigelow Laboratory for Ocean Science
East Boothbay, Maine
http://www.bigelow.org/
https://eco.bigelow.org
-- 
Ben Tupper Bigelow Laboratory for Ocean Sciences https://www.bigelow.org/
Tick forecast https://eco.bigelow.org/

	[[alternative HTML version deleted]]