Gradual Net

Abstract

Convolutional neural network (CNN) is mainly composed of convolution, pooling, and non-linear activation layers. Nowadays, almost all networks use only $2 \times 2$ max pooling or convolution layers with stride of 2 for down-sampling. This technique is known to be good at extracting good feature, but it also has the constraint that feature map size is always reduced dramatically to half. In this work, we propose a simple new sampling technique that we call non-integer strided sampling (NSS), which enables free feature map size change, so that it is not always reduced to half. Using this NSS layer, we design a new type of network architecture, GradualNet, which makes the feature map size change more smoothly than it is in existing networks.Our results showed that NSS can improve the performance of networks without having more parameters. Especially, it shows 1.82% accuracy improvement with CIFAR-100 without data augmentation compared to the baseline ResNet. Moreover, we propose other interesting possibilities for a CNN architecture based on the NSS layer. The results revealed that previous networks have been stuck in a stereotype, and this could be an important discovery in CNN architecture that has the potential to resolve this stereotype.