Automatic detection and identification of object by fusing camera and marine radar measurements

Abstract

As research on unmanned systems has been actively conducted recently, technology that recognizes obstacles and the surrounding environment, which is required for performing various tasks effectively, has become important. In maritime environments, radar has been used as a primary sensor to detect objects for navigation and collision avoidance, but recently, cameras are also being considered to improve the reliability and performance of detection and to perform it automatically. This study addresses active detection and identification by matching the relative position of floating objects detected by radar images and ships detected in camera images. First, convolutional neural networks are used to detect ships from camera images and to semantically classify marine radar images into floating object, noise, and land. Then, a newly developed robust data association algorithm is applied, using parameters representing the correlation between two sensor measurements. The performance of the proposed algorithm is validated using a camera and radar dataset obtained in real maritime environments.