Performance improvement of densely connected convolutional neural network by using exponentially increasing feature dimension

Abstract

Object recognition and classification is a basic and essential part of computer vision. As deep convolutional neural networks emerged, there was huge success in object recognition and classification tasks. However, classification in datasets with many classes (over 100 classes) is still challenging. Examples include ImageNet 1K dataset and Cifar100. To improve performance in datasets with many classes, many approaches have been proposed. Regularization techniques[11,12,13,14], structural approaches[1,2,3,4,5,6,7], data augmentation[15, 16], and new activation functions[17,18,19] have been proposed. In this paper I take a structural approach. In recent years many structures have been proposed: LeNet[6], AlexNet[7], GoogleNet[5], Residual Networks(ResNet)[3] Pyramidal Residual Networks(Pyramid ResNet)[1] and Densely Connected Convolutional Neural Networks(DenseNet)[2]. Among these structures, Pyramid ResNet and DenseNet are the current state of the art. In particular, the Densely Connected Convolutional Network is an efficient structure in number of parameters and computational cost. Though DenseNet is very efficient in parameters and computation complexity, recently it has become known that there exist inefficiencies. In this paper I focus on inefficiencies in DenseNet. I investigate problems in linearly increasing input feature dimensions in DenseNet. To solve this problem, I propose a structural approach that exponentially increases input feature dimension of units as unit index increases.” Experiments on Cifar100 show that proposed structure has the same or higher recognition accuracy than Pyramid ResNet and DenseNet for identical number of parameters (0.8M and 1.7M) with 1/2 to 1/3 computational cost.