Paper: A Technique for Mixed Reality Remote Collaboration using 360 Panoramas in 3D Reconstructed Scenes
This paper has received two awards to date:
- Best Full Paper – Runner Up at the 25th ACM Symposium on Virtual Reality Software and Technology
- Honorable Mention Award at the ACM CHI 2019, a top Conference on Human Factors in Computing Systems
Mixed Reality (MR) remote collaboration provides an enhanced immersive experience where a remote user can provide verbal and nonverbal assistance to a local user to increase the efficiency and performance of the collaboration.
This is usually achieved by sharing the local user’s environment through live 360 video or a 3D scene, and using visual cues to gesture or point at real objects allowing for better understanding and collaborative task performance. While most of prior work used one of the methods to capture the surrounding environment, there may be situations where users have to choose between using 360 panoramas or 3D scene reconstruction to collaborate, as each have unique benefits and limitations.
In this paper we designed a prototype system that combines 360 panoramas into a 3D scene to introduce a novel way for users to interact and collaborate with each other. We evaluated the prototype through a user study which compared the usability and performance of our proposed approach to live 360 video collaborative system, and we found that participants enjoyed using different ways to access the local user’s environment although it took them longer time to learn to use our system.
We also collected subjective feedback for future improvements and provide directions for future research.
Theophilus Teo, Ashkan F. Hayati, Gun A. Lee, Mark Billinghurst, and Matt Adcock. 2019. A Technique for Mixed Reality Remote Collaboration using 360 Panoramas in 3D Reconstructed Scenes. In 25th ACM Symposium on Virtual Reality Software and Technology (VRST ’19). ACM, New York, NY, USA, Article 23, 11 pages. DOI: https://doi.org/10.1145/3359996.3364238
Figure 1: Prototype system overview demonstrating how the local user connects with the remote user to share visual cues to collaborate.
Download the full paper here.
For more information, contact us.