All-day 3D object detection using image to image translation

Abstract

It is very important to detect objects in autonomous driving situations. However, it will be difficult to detect objects in certain situations where the surroundings are dark, fog, or rain. Among such specific situations, I propose a model that enables object detection both day and night. Since most of the existing 3D driving data have many daytime conditions, it is difficult to detect an object at night. So, using the image conversion model, the corresponding day image is converted into a night image based on the night image that is not paired with the day image. At this time, the depth map was used as a guide so that it could be more helpful in image conversion. In the 3D object detection part, depth aware convolution is used that considers the characteristics of the driving image. In addition, corner loss was applied to correct the vehicle heading angle, and the image translation model and the object detector model were fine-tuned to each other.