Paper: A Technique for Mixed Reality Remote Collaboration using 360 Panoramas in 3D Reconstructed Scenes

November 15th, 2019

This paper has received two awards to date:

Mixed Reality (MR) remote collaboration provides an enhanced immersive experience where a remote user can provide verbal and nonverbal assistance to a local user to increase the efficiency and performance of the collaboration.

This is usually achieved by sharing the local user’s environment through live 360 video or a 3D scene, and using visual cues to gesture or point at real objects allowing for better understanding and collaborative task performance. While most of prior work used one of the methods to capture the surrounding environment, there may be situations where users have to choose between using 360 panoramas or 3D scene reconstruction to collaborate, as each have unique benefits and limitations.

In this paper we designed a prototype system that combines 360 panoramas into a 3D scene to introduce a novel way for users to interact and collaborate with each other. We evaluated the prototype through a user study which compared the usability and performance of our proposed approach to live 360 video collaborative system, and we found that participants enjoyed using different ways to access the local user’s environment although it took them longer time to learn to use our system.

We also collected subjective feedback for future improvements and provide directions for future research.

Theophilus Teo, Ashkan F. Hayati, Gun A. Lee, Mark Billinghurst, and Matt Adcock. 2019. A Technique for Mixed Reality Remote Collaboration using 360 Panoramas in 3D Reconstructed Scenes. In 25th ACM Symposium on Virtual Reality Software and Technology (VRST ’19). ACM, New York, NY, USA, Article 23, 11 pages. DOI:

Figure 1: Prototype system overview demonstrating how the local user connects with the remote user to share visual cues to collaborate.

Download the full paper here.

For more information, contact us.

Subscribe to our News via Email

Enter your email address to subscribe and receive notifications of new posts by email.