Removal of reflected virtual images in visual recognition utilizing 3D depth information

Abstract

We address removal of reflected virtual images in visual recognition system. Images reflected by mirror are similar to real object, so it is non-trivial task to differentiate them. Conventional object detectors, which do not deal with this problem, obviously recognize reflected image as real object. We propose a new method for elimination of reflected virtual images utilizing 3D depth information. The proposed method compares relationship between spatial information of environment where an object exists and detected object in 3D space to discriminate mirror reflection image. As a representative case of object detection performance degradation caused by mirror reflection, we selected a detecting people in indoor environment as target system to solve the problem. We propose a deep learning based method that detects layouts of indoor environment using semantic segmentation and plane detection. We also propose a method of processing the reflected image using the relationship between the detected layout and the 3D coordinates of the person candidates. To verify the proposed method, a large dataset was obtained in a real world environment. The performance of the algorithm is verified by comparing conventional detector with proposed method in the obtained dataset.