Augmented World Models

Augmented World Models

Data61 are developing novel methods for interpreting and fusing data from different sensor modalities. We focus on augmenting 3D data for enhanced understanding of scenes, quantifying the world and collecting data beyond “”our eyes””. Examples of some of the 3D++ technologies we developed include:

• Colour Zeb: 3D Lidar + RGB. Broad application in the 3D mapping area.
• HeatWave: 3D + Thermal + RGB. We are exploring applications in medicine, thermal mapping for manufacturing, construction and infrastructure surveys, and disaster assessment and recovery.
• 3D + Gas: Can be used for mapping of gas emissions and plumes in indoor/underground structures (LNG plants, mines, etc.) and outdoor environments (coal seam gas fugitive emissions mapping, airborne pollution assessments, etc.).
• In-Situ Hyperspectral Mapping (ISHM): 3D + Hyperspectral + RGB + Thermal. This technology has been identified by other CSIRO Business Units (Agriculture and Biosecurity) as essential for phenotyping assessment and identification of pest and diseases. One of our systems, AgScan3D+ (which combines 3D + Hyperspectral + RGB), is being used in joints projects with QLD DAF funded by HIA, projects funded by Vinyculture Australia, phenotyping projects funded by CSIRO Agriculture Business Unit.

Data61 are world leading in combining high-accuracy 3D data with other sensor modalities. We have created a new area in robotics and computer vision, called Augmented World Models or 3D++. While a few other groups have done some work on Lidar + RGB, we are developing a comprehensive set of sensor fusion algorithms. These algorithms allow us to generate world models that combine 3D lidar models with data obtained from RGB, thermal, hyperspectral, environmental, gas, and many other sensors. Data61 are also developing algorithms to add semantics to 3D scenes, including geometric (CAD) models, semantic labels, automatic spectral clustering, etc.