Online multi-object tracking using deep neural network

Abstract

In computer vision research, various algorithms using deep neural networks (DNNs) have been developed. Among the deep neural networks, the convolutional neural networks (CNNs) have a structure that expresses the spatial characteristics of the images well, and the recurrent neural networks (RNNs) have a structure that can utilize the information on the sequential images well. Using this characteristics, we can apply CNNs and RNNs to visual multi-object tracking (MOT) research using image as input. In order to apply the DNNs to the online MOT process, an appropriate neural network structure is needed to solve the problems that may occur when considering multiple objects simultaneously. In this dissertation, we propose an online MOT system that combines the CNNs and the RNNs. Multiple object tracking systems focus on distinguishing between individual objects within the same object class. Therefore, neural networks should be trained to extract features that make it easier to distinguish minute differences between objects using visual information of images. To do this, we generate the discriminative appearance feature using triplet convolutional neural networks (triplet-CNN). In addition, the RNNs are used to estimate the motion models of the objects based on the sequential information of the target objects, and the Long Short Term Memory (LSTM) neural networks with enhanced memory function are used for data association between trajectories and target objects. To improve the performance of the proposed system, we introduce the generative adversarial network (GAN) structure for data augmentation. Using data augmentation, we can improve the performance of the triplet-CNN, which leads to the performance improvement of the proposed MOT system. We also propose a novel gradient descent algorithm called a rejuvenating adaptive PID-type optimizer (RAPIDO) for fast deep learning. RAPIDO algorithm adapts over time and performs optimization using current, past and future information similar to the PID controller. It is suited for optimizing deep neural networks that consist of activation functions such as sigmoid, hyperbolic tangent and ReLU functions because it can adapt appropriately to sudden changes in gradients.