Online image enhancement for robust visual perception in hazy environment

Abstract

For autonomous robots and vehicles, it is essential to recognize the surrounding environment accurately. In particular, visual information is the most intuitive and informative for autonomous navigation or human-robot interaction. However, visibility is easily corrupted in water or disaster environments where the robot operation is necessary. This low visibility condition causes the failure of existing robot algorithms. Therefore, an image quality improvement method called dehaizng or image enhancement is essential. In this dissertation, we propose image enhancement methods for robust visual perception. The proposed methods are suitable for various low visibility environments and verified on qualitative, quantitative, and computer vision applications. The technical and theoretical summary of this thesis is as follows. First, we proposed the image enhancement method to improve the performance of localization and place recognition in the underwater environment. In the marine environment where the turbidity of the surrounding medium is high, such as underwater, image attenuation, and distortion occur severely. Especially when the robot is operated underwater for a long time, such as hull inspection, stable visual information is necessary. Such image attenuation causes a severe problem in the robot operation. In this study, we introduce a new image degradation model, including different attenuation and distortion effects, and propose the image enhancement method by predicting image attenuation parameters through sparse distance information and illumination pattern prediction. Second, we developed the single image restoration and enhancement method that combines the advantages of a model-based method and a fusion-based method. With multi-band decomposition, we construct an intensity module for restoration and a Laplacian module for detail enhancement. The intensity module estimates ambient light and transmission map without any haze-relevant priors, and the non-linear mapping function is applied for image details. The proposed method is available for both color and grayscale images uniformly. This study verified the performance of various robot vision algorithms such as robot position recognition and semantic segmentation. Third, we proposed a learning-based image dehazing method for color-distorted images from underwater. Since light is attenuated with respect to the wavelength, underwater images have biased colors such as bluish, greenish, or yellowish. Also, color distortion is challenging to generalize from the conventional haze image model. To restore underwater images with color correction, we developed unpaired image training networks with real underwater and clear images. The networks are trained with multi-objective losses that capture image color, details, and textures. Last, we investigated the limitation of the single scattering image model used in the image processing field by analyzing real haze and clear images. We evaluated the proposed and previous methods on real hazy images that have a natural scene with generated fog by real fog machines. Each scene has a ground-truth clear image and degraded images, and we were able to analyze the characteristics of image degradation and the limitation of previous enhancement methods. Also, we examined the characteristics of each method by integrating and analyzing various methodologies proposed in this study. Through this, we applied the proposed approach to the images under different hazy situations such as underwater and disaster environment, so that the appropriate algorithm could be identified according to the situation.