Exploring pathways for digital delivery of environmental datasets via APIs.
The Challenge (Trigger/Issue)
The CSIRO Data Access Portal (DAP) enables public access to various types of datasets published by CSIRO scientists. The continental scale geospatial datasets hosted on the DAP usually provides a medium for scientists to easily access big datasets for research via direct download. However, users that are interested in data access at regional to paddock scales; and who are limited by time and computational resources; often find it difficult to access and use datasets hosted on the DAP. Hence, in such cases, data inaccessibility becomes a major roadblock for maximising impact from CSIRO science.
Our response (Opportunity)
Recent emergence of APIs (Application Programming Interface) has enabled easy accessibility and delivery of digital datasets in various sectors. This project aims to explore the existing API pathways for potential data accessibility and subsequent smooth delivery for end users wanting to use CSIRO datasets to inform local to regional scale management decisions. To achieve this objective, this project set out to test the pathways for data delivery for end users such as Ag Tech companies and government agencies that need a quick assessment of paddock and regional scale environmental information for decision making and reporting. To address this goal, this project started with three different API pathways (productization and technical implementation) and three different environmental datasets developed by CSIRO scientists. Depending on the technological readiness of the API pathways and environmental datasets chosen, this project proceeded with one pathway and two environmental datasets for data delivery.
Results (Impact)
This report has two aims (i) provide a critical overview of the available API pathways and their pros/cons for CSIRO Environment scientists who have a need to deliver spatial data products via APIs and (ii) deliver HCAS and BAR datasets via APIs mounted on the AWS platform
Additional information
Two APIs have been developed that are additional deliverables of this project.
Below is a diagram illustrating the end-to-end system developed.
For more information please contact peter.fitch@csiro.au.