Robust image processing and camera control for variation of light condition and its application to visual navigation

Abstract

In the operation of the robot, the technology of recognizing the position and the surrounding environment is very important. In particular, the camera is one of the most widely used sensors in the technology to recognize the position and the environment. However, the camera is sensitive to changes in light conditions or the surrounding environment. Loss of image occurs in a rapidly changing environment or high dynamic range environment. This image loss eventually leads to failure of the visualization-based navigation system due to the robot's position and environmental awareness errors. In order to overcome this problem, research is needed to restore these losses through pre or post image processing. Another approach is to control camera parameters (aperture, exposure time, gain) to minimize these image losses. This thesis introduces a method to secure the performance of vision-based robot navigation algorithm by restoring or minimizing image loss. Algorithm suitable for extreme environment as well as general environment was proposed, and value was veried through actual experiments under various conditions and environments. The technical summary of this dissertation is as follows. First, we propose a method to improve the performance of vision-based navigation algorithm by composing two images with dierent exposure time. In a space where indoor and outdoor coexist, image loss occurs frequently due to the dierence in illuminance of articial light and light from the outside. Through this study, we analyze the image characteristics lost by illumination and light, and propose a loss area detection algorithm. In addition, we introduced a method of restoring only saturated areas, not entire pixels, by compositing two images with dierent exposure values. The images were acquired and veried in partially over-exposed and under-exposed environments. Second, based on the previous research, we proposed a camera exposure time control algorithm that minimizes the saturation region. Instead of the automatic exposure control that determines the exposure value according to the amount of light coming from the camera, we proposed a method of analyzing the image in advance and determining the optimal exposure time of the image. Image metric is dened using gradient to measure the amount of image information for image analysis and local entropy to measure the amount of image loss. And Bayesian Optimization, a machine learning based optimization method, was used to estimate the global optimal value quickly and accurately through minimal training process. Third, in order to control exposure time and gain at the same time, we redene the metric considering noise. In addition, in order to secure real-time performance, synthetic images were created using the camera response function and gain equation, not the actual image, and the image metric was analyzed to extend it to a 2-dimensional Bayesian optimization algorithm. The proposed algorithm was veried through actual experiments in indoor, outdoor, and dynamic environments. The proposed method guarantees real-time performance by supporting a rate of 20 40Hz. Finally, studies were conducted on low-light environments in contrast to previous studies focused on over-exposure or High Dynamic Range (HDR) environments. In a low-light environment, it is generally necessary to increase the exposure time or gain, but negative factors such as motion blur or noise exist to limit the exposures. In this study, high-exposure synthetic images are created with one actual image and fused to create high-exposure image that minimize negative factors in low-light environments. The proposed algorithm was veried through actual experiments in indoor and outdoor dark environments.