(A) study on deep neural networks of single image super-resolution and stereo vision via similarities between spatially scattered features

Abstract

An autonomous vehicle can be realized only by developing a reliable environment recognition system by processing information obtained through various sensors. This dissertation is a study on the environmental recognition system using a camera, and it includes single image super-resolution and stereo vision. The environmental recognition system using only the camera sensor lacks the amount of information, and research is demanded to complement and solve the insufficient information. The development of convolutional neural networks (CNN) has remarkably improved the current research on single image super-resolution (SISR). Several high-quality studies have been performed on reconstruction accuracy and perceptual quality, which are the two main issues in SISR. Nevertheless, numerous problems in SISR remain unsolved. SISR is inherently an ill-posed problem owing to insufficient information, and as the scale factor increases, the lack of information becomes even more pronounced. We have studied ways to solve the local characteristics of CNN to deal with additional useful information. A CNN uses a convolution layer designed based on local features, and repeatedly accumulates these features to expand a receptive field. We have explored network structures that can directly handle global information even at lower layers, which are not covered by the receptive field of a CNN. In this paper, we propose a non-local attention SISR network that generates and utilizes the globally scattered similarity information of features. In addition, we propose a very deep architecture based on dense blocks that does not suffer from gradient vanishing without any normalization. Experimental results on standard benchmark datasets indicate the effectiveness of the proposed network, which exhibits state-of-the-art performance in terms of reconstruction accuracy and perceptual quality. The application of additional useful information is also effective in stereo vision. The proposed non-local attention-based network can directly access global information and provide features that general convolutional neural networks cannot handle. Although the conventional networks estimate the disparity of a stereo image by using the relationship between the surrounding information, the proposed network has a difference in using additional global information. The proposed non-local attention-based network was optimized and applied, and the improvement of disparity estimation accuracy was demonstrated through the experimental results on the benchmark database. The proposed non-local attention-based network was optimized and applied to the stereo vision network, and experimental results on standard benchmark datasets indicate the effectiveness of the proposed network.