Memory access reduction techniques for CNN accelerating systems

Abstract

Convolutional neural networks (CNNs) have been actively applied to computer vision applications using deep-layered architecture in order to achieve high accuracy. However, the deep-layered architecture increases energy consumption required in implementing CNN accelerating systems, so reducing memory access that consumes large energy is important to achieve high energy efficiency. The computer vision applications can be classified into image-level labeling application, such as image classification and object detection, and pixel-level labeling application, such as super-resolution and image-to-image translation. This dissertation proposes memory reduction techniques and hardware architecture for both applications by analyzing unique characteristics of each application. In the image-level labeling application, an input image is processed by a feature extraction network to generate inferences for objects in the image. In the feature extraction network, the number of features involved in the convolution of a shallow layer is larger than that of kernels. Due to the feature extraction network, however, the number of features decreases while that of kernels increases as the layer deepens. By taking into account the numbers of data, a hybrid convolution technique, which selects either kernel-stay convolution or feature-stay convolution, is proposed to reduce memory access. In the pixel-level labeling application, on the other hand, an input image is translated to another image of the same or higher resolution by using a encoder-decoder network that infers every pixel. In the encoder-decoder network, the number of features is maintained even if the layer deepens. As high-resolution images are becoming mainstream, in addition, the number of features generated is very large. To reduce memory access, CNN compression and layer-chaining convolution techniques are proposed. Realizing the proposed techniques, a neural processing unit that accelerates CNNs is designed for each application, and CNN accelerating systems have been implemented.