Adaptive convolutional neural network structure through gradient-based learning

Abstract

A design of convolutional neural network (CNN) architectures has become essential for computer vision and machine learning. There are various ways to determine the best CNN structure for adapting to different tasks and datasets. However, most of the CNN structures are still designed by heuristics or professional opinions. In this work, we propose an adaptive CNN which data-dependently searches for the effective size of filters and its distribution. We demonstrate that the filter sizes used in deep CNNs can also be optimized through gradient-based learning. The main idea to enable the adaptive CNN to automatically allocate of resources over different filter sizes. The proposed adaptive CNN has a mask, which enables the automatic allocation of resources over different filter sizes. The mask is differentiable and can be optimized through usual gradient-based methods with the weights simultaneously. Most importantly, the proposed adaptive CNN generalizes both a single-path structure and a multi-path structure. The adaptive learning of the convolutional structure is evidenced by several benchmarks, compared with various structures, all of which have the same depth and number of channels, but different filter sizes.