Automatic acoustic-material estimation of indoor environments for realistic sound generation

Abstract

Generating a realistic sound is one of the essential components in various research and applications, such as immersive AR/VR and voice recognition, that require the sound of the real environment. The 3-D structure of the real environment and the acoustic material properties are necessary to generate a realistic sound. Acoustic experts can obtain the acoustic properties of the real environment by utilizing specific methods comprising specialized equipment and conditions with specialized knowledge. However, it is difficult for general users to obtain acoustic material information from the real environment, making it challenging to generate realistic sounds. This dissertation proposes an integrated framework that can automatically measure the acoustic properties of an indoor environment without complex equipment and specialized knowledge for realistic sound generation. Specifically, we propose a method for automatically identifying the type of acoustic material from 3-D point cloud data to measure acoustic properties in an indoor environment. Furthermore, we propose a simple measurement method based on signal separation and deep learning using an inexpensive device to estimate the acoustic properties of identified materials. At the end of the dissertation, we integrate the proposed methods into a mobile robot framework to show that our proposed method can automatically estimate the acoustic properties of a real 3-D environment and generate realistic sounds.