Space-adaptive mutual space generation for mixed reality remote collaboration

Abstract

To achieve a sense of coexistence in Mixed Reality (MR) remote collaboration where Augmented Reality (AR) hosts and Virtual Reality (VR) clients are leveraged, it is necessary to overcome spatial differences between spaces and create a mutual space with a unified coordinate system. This work proposed a space-adjusting technique with redirected walking and multiple spatial matching algorithms to create a spatial affordance-aware mutual space from heterogeneous remote spaces where users could physically interact in connected virtual space. In the beginning, Study 1 determined the threshold range of relative translation gain (RTG) in redirected walking with two user studies (n = 64) and revealed that three components of spatial configuration, space size, object existence, and spatial layout, all significantly affected the users' visual perceptual RTG thresholds and estimated the allowable space adjustment constraints according to the spatial configuration. In the following, Study 2 proposed an edge-centric physical space rescaling using RTG as a space rescaling term and spatial registration algorithm to utilize basic geometric information from users' physical space in the virtual space. The evaluation results showed that RTG-based space registration could not only align planes and edges most but also secure the largest interactable area compared to other spatial registration methods. Study 3 proposed a method for extracting and matching interactable subspaces from multiple disparate remote spaces and generating mutual spaces using an optimization algorithm that determines the optimal initial positions of users. Experiments on 900 space combinations generated by varying the number of client spaces to two, four, and six proved that the spatial affordance-aware subspace allocation method could generate a mutual space suitable for collaborative contexts even if the number of client spaces increases. Finally, the proposed mutual space generation algorithm was implemented in the MR remote collaboration system, and mutual space generation guidelines for developers were derived. The results of the three phases of research showed that physical space transformation using translation gain-based redirected walking is effective in aligning heterogeneous spaces, and the spatial affordance-aware interactable subspace allocation approach is beneficial to generating a mutual space from multiple dissimilar spaces. The findings from the three-stage study will enable remote collaboration between distant users as if they are all in the same room, connecting people across spatial boundaries and contributing to the widespread adoption of remote work, immersive VR content creation, and addressing urban sprawl and rural decline.