[R-sig-Geo] Climate data in R
Isaque Daniel
isaquedanielre at hotmail.com
Wed Aug 3 01:02:49 CEST 2016
Incredible dataset sources Tom!
Best
Isaque
------------------------------------------------------------------------------------------------------------------
Eng. Agr. Isaque Daniel Rocha Eberhardt
Mestre em Sensoriamento Remoto - Instituto Nacional de Pesquisas Espaciais (INPE)
Doutorando em Transportes - Universidade de Bras??lia (UNB)
Mobile: +55 (061) 99015658
------------------------------------------------------------------------------------------------------------------
Agronomist engineer
Master in Remote Sensing - National Institute for Space Research (INPE) - Brazil
PHD Student in Transport - Bras??lia University (UNB)
________________________________
De: Tom Philippi <tephilippi at gmail.com>
Enviado: terça-feira, 2 de agosto de 2016 21:34
Para: Isaque Daniel
Cc: Kenny Bell; M. Edward (Ed) Borasky; R-sig-geo mailing list
Assunto: Re: [R-sig-Geo] Climate data in R
Without knowing what part of the globe you need climate data for, nor what data you need (30 year "normals", bioclim parameters, monthly values, daily, extremes?), I can't give a useful specific answer.
What follows is an initial draft of a web page on climate data in R. It's somewhat US and North America centric.
The number of APIs providing climate data is growing rapidly. The downside is that most data providers are region/continent specific. NOAA NDCD, ACIS, Mesowest, PRISM, SNOTEL, SNODAS, and others are North America or CONUS, there are different APIs and packages for EU, UK, NZ, and Canadian climate. The few worldwide services tend to have lower station densities than region-specific services, especially for older dates.
Some APIs and overviews:
ACIS (NOAA): http://www.rcc-acis.org/docs_webservices.html
Web Services - RCC-ACIS<http://www.rcc-acis.org/docs_webservices.html>
www.rcc-acis.org
Introduction. The Applied Climate Information System (ACIS) Web Services consists of five types of calls - StnMeta, StnData, MultiStnData, GridData and General.
NOAA Co-ops: http://tidesandcurrents.noaa.gov/api/
PRISM http://www.prism.oregonstate.edu/index.phtml
MesoWest: http://www.rcc-acis.org/docs_webservices.html http://synopticlabs.org/api/
http://www.esrl.noaa.gov/psd/data/gridded/
https://climatedataguide.ucar.edu/climate-data/global-temperature-data-sets-overview-comparison-table
http://www.cgiar-csi.org/data/uea-cru-ts-v3-10-01-historic-climate-database
All of those APIs can be hit from R; some have R packages available to make it easier. [And, more importantly for my use cases, those packages isolate my code that hits the package functions from changes in the underlying API. Instead of each of us having to keep up with changing APIs and fixing our broken code every 6-12 months, package maintainers put out new versions that work with the new APIs but use the same parameters and return the same objects. If my code from last year fails to fetch data this year, I usually can simply update the package. Thanks, maintainers!]
For R packages, start with https://github.com/ropensci/opendata#climate and http://ropensci.org/packages/
Most non-raster APIs have 2 steps: a station finder that you query by name or location, then a data fetcher pulling data from the found stations.
Raster datasets tend to be huge, and tend to be organized as 1 file per date. There's a PRISM API that lets you select locations and retrieve data for a range of dates for requested rectangular AOAs.
My 2 lessons learned as an ecologist using climate data (not a real climatologist) are:
1: Spatial interpolation is non-trivial. PRISM uses not just distances from stations of record, but also topography: elevation for temperatures and precipitation and side of the mountain (very important for precipitation). Don't just use the nearest station or inverse-distance krige unless you are in a broad flat plain or steppe. [Sorry, Ed!] There are a couple of raster products for daily data (not just the monthly PRISM), but they tend to be available a few weeks after the fact.
2: Station records are instrumentation values, not actual temperatures & precipitation amounts. Almost all climate data providers QA and filter the values, because sensors drift and fail. However, the appropriate amount of filtering is dependent on your use. Some blips in the raw data are instrumentation errors, some are short extreme events; some are of a magnitude & duration where it can't be determined which of those they are. The appropriate or optimal amount of filtering for uses such as crop production models is much greater than the appropriate amount for some ecological applications where short intense events can be important. Even within larger agencies like NOAA, different regional climatologists filter the data to different extents. To do valid science, if extreme events matter for your use, you need to know the preprocessing on your datastream, lest you get fooled into thinking that region-specific filtering is climatological signal.
If anyone has comments, additions, or corrections to the above, please send them to me!
If the OP has a specific data type in a specific region, I might have a specific suggestion.
Tom 2
Tom Philippi
Quantitative Ecologist & Data Therapist
Inventory and Monitoring Program
US National Park Service
On Mon, Aug 1, 2016 at 12:49 PM, Isaque Daniel <isaquedanielre at hotmail.com<mailto:isaquedanielre at hotmail.com>> wrote:
Hi,
You can follow the example in: http://stackoverflow.com/questions/34619218/extract-raster-values-from-stack-to-points-in-for-loop
Is really simple, using coordinates and extract functions of raster package.
Best
Isaque
------------------------------------------------------------------------------------------------------------------
Eng. Agr. Isaque Daniel Rocha Eberhardt
Mestre em Sensoriamento Remoto - Instituto Nacional de Pesquisas Espaciais (INPE)
Doutorando em Transportes - Universidade de Bras??lia (UNB)
Mobile: +55 (061) 99015658<tel:%2B55%20%28061%29%2099015658>
------------------------------------------------------------------------------------------------------------------
Agronomist engineer
Master in Remote Sensing - National Institute for Space Research (INPE) - Brazil
PHD Student in Transport - Bras??lia University (UNB)
________________________________
De: R-sig-Geo <r-sig-geo-bounces at r-project.org<mailto:r-sig-geo-bounces at r-project.org>> em nome de Kenny Bell <kmb56 at berkeley.edu<mailto:kmb56 at berkeley.edu>>
Enviado: segunda-feira, 1 de agosto de 2016 19:47
Para: M. Edward (Ed) Borasky
Cc: R-sig-geo mailing list
Assunto: Re: [R-sig-Geo] Climate data in R
See https://cran.r-project.org/web/packages/prism/index.html for
CRAN - Package prism<https://cran.r-project.org/web/packages/prism/index.html>
cran.r-project.org<http://cran.r-project.org>
prism: Access Data from the Oregon State Prism Climate Project. Allows users to access the Oregon State Prism climate data. Using the web service API data can easily ...
interpolated US weather data
On Mon, Aug 1, 2016 at 12:43 PM, M. Edward (Ed) Borasky <znmeb at znmeb.net<mailto:znmeb at znmeb.net>>
wrote:
> I have a similar question. Assume I've downloaded the climate data for all
> the stations in the US state of Oregon, including their latitude, longitude
> and elevation, using rnoaa. How do I *interpolate* climate values for an
> arbitrary latitude / longitude / elevation inside a triangle defined by the
> nearest stations?
>
> On Mon, Aug 1, 2016 at 3:24 AM Miluji Sb <milujisb at gmail.com<mailto:milujisb at gmail.com>> wrote:
>
> > Dear all,
> >
> > I have a set of coordinates. Is it possible to extract climate data
> > (temperature and precipitation) by coordinates using the R packages such
> as
> > rnoaa?
> >
> > For example;
> >
> > out <- ncdc(datasetid='ANNUAL', stationid='GHCND:USW00014895',
> > datatypeid='TEMP')
> >
> > But instead of stationid can I pass a list of coordinates through it?
> > Thanks a lot!
> >
> > Sincerely,
> >
> > Milu
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo at r-project.org<mailto:R-sig-Geo at r-project.org>
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
> --
> How many people can stand on the shoulders of a giant before the giant
> collapses?
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org<mailto:R-sig-Geo at r-project.org>
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Kendon Bell
Email: kmb56 at berkeley.edu<mailto:kmb56 at berkeley.edu>
Phone: (510) 612-3375<tel:%28510%29%20612-3375>
Ph.D. Candidate
Department of Agricultural & Resource Economics
University of California, Berkeley
[[alternative HTML version deleted]]
_______________________________________________
R-sig-Geo mailing list
R-sig-Geo at r-project.org<mailto:R-sig-Geo at r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
[[alternative HTML version deleted]]
_______________________________________________
R-sig-Geo mailing list
R-sig-Geo at r-project.org<mailto:R-sig-Geo at r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
[[alternative HTML version deleted]]
More information about the R-sig-Geo
mailing list