Senaps: data science to data service
Senaps is a cloud platform which integrates real-time sensing, spatial data and composable analytics into digital services delivered by web APIs.
We are developing technology to help Australia realise the $315b digital innovation opportunity. Bringing data driven science to market is both expensive and challenging. Applications which bring together real-time sensing, spatial data and complex analytics are no exception to that challenge. Senaps provides a integrated spatio-temporal data platform help data driven applications reach their customers. Senaps broadly includes three major components:
- Data services for sensor data
- Data services for geospatial data
- Scientific workflows
The platform combines these three major components as an integrated multi-tenanted cloud platform.
The Senaps Analysis Service combines data-driven workflow concepts with containerisation and serverless computing to provide a cloud based scientific workflow system. The Analysis Service allows users of Senaps to deploy their own analytics and share this with other users.
- Generic run-time environment provide maximum flexibility for combining existing and new software models written in nearly any language into complex workflows.
- ‘Serverless’ model allows developers to avoid managing infrastructure
- Struct typing and interfaces between models enables composability and sharing of analytics.
- Open web APIs allow for easy integration into modern web software development.
Sensor Data Service
Senaps incorporates a device and communications agnostic sensor data management service. Anything from oyster heart signals to daily weather stations readings can be managed. Interoperability between data streams is enabled by a common information model allowing composable analytics to be ‘connected’ to any sensor data stream. With a pragmatic but strict metadata model leveraging shared vocabularies, sensor data sources can be integrated into applications without a strong software coupling. This help breakdown the tight vertical integration between sensors and user facing App often found within the IoT sector.
Spatial Data Services
Senaps also provides integration with standardised spatial data web services and hosts the THREDDS Data Server. THREDDS provides services for making netCDF multidimensional array data available via web services. The integrated THREDDS service is augmented with the Senaps authorisation and sharing model and also a dedicated API for uploading new datasets. The spatial data services are then also integrated as a key component many analytics workflows.
The Graincast App provides grain farmers with daily estimated soil moisture and yield potential for any location in Australia. Senaps provides the underlying data and analytics management for the Graincast App. Graincast heavily leverages the Analysis Service to run 10s of thousands of APSim simulations every night for Graincast users across Australia. Predictions are provided to the Graincast user interface by APSim which is deployed as a model within Senaps and integrated with climate and soils data services. The APSim model is an internationally recognised crop simulation system. Near real-time daily climate data sourced from SILO and BoM is also managed by Senaps.
The Digiscape Waterwise project is using to deliver irrigation analytics to growers. The project is using Senaps to combine real-time sensor data, BoM forecasts and analytics.
Researchers from the Waterwise team embedded their existing analytics code (written in R programming language) into Senaps workflows. Real-time data from canopy temperature sensors and weather forecasts from BoM are integrated into the analytics workflow.
The user can see the cumulative heat stress of plants in between rain or irrigation events. The Waterwise planner makes a recommendation to irrigate when the heat stress rises above the threshold.
Senaps is being used in many more varied and interesting ways:
- The underlying ICT platform for the Digiscape FSP
- Storing animal tracking data
- To support climate and hydrology data for the Basin Futures project
- Storing data and analysing heart rate from Oyster biosensors
- Climate data history for any location in Australia
- Hydraspectra water quality sensing analytics workflow
- the Sense-T data platform