Natural Communication with Spot : A Vacation Student Project

Nov 2022 – Feb 2023

During the summer 2022, Hashini led a vacation student project named “Fetch Spot: Natural Communication in Human-Robot Teams” with five other supervisors from Robotics and Autonomous Systems group; Pavan Sikka, Fletcher Talbot, Alex Pitt, Jason Williams and David Howard. Four students in this project, Callum Bennie, Bridget Cassey, Annabelle Nott and Ze’ev Krischer, implemented speech, haptics, gesture and AR-based communication with a Spot robot. We aim to extend this project to study types of interaction modalities suitable for communicating diverse information to human collaborators working with multiple robots to achieve optimal dynamic situational awareness.

Note: The idea for diversity-targeted positions within this project was co-developed with Robogals Brisbane chapter. 

Project Outcomes

• 15 Jun 2024: The natural speech component was tested within a space exploration demonstration, and we published the results in a paper titled “Demonstrating Event-Triggered Investigation and Sample Collection for Human Scientists using Field Robots and Large Foundation Models” at the RSS conference
  
  
• 2 Dec 2023: We published a workshop paper titled “Alternative Interfaces for Human-initiated Natural Language Communication and Robot-initiated Haptic Feedback: Towards Better Situational Awareness in Human-Robot Collaboration” at OzCHI conference
  
  
• 7 Jun 2023: Callum demonstrated the natural speech communication component at CSIRO MARS conference