VLSI architecture for deep convolutional neural networks

Abstract

In recent decades, the neural network-based inference algorithms have become one integral part in nearly all the recognition systems due to its high recognition accuracy. As the algorithms implemented in recent recognition systems can achieve better recognition accuracy at the expense of large computational complexity, it is important to devise energy-efficient hardware architectures specialized for the neural network operations. Among various algorithms, this dissertation is devoted to developing efficient VLSI architectures for convolution operations of convolutional neural networks (CNNs) which have been widely utilized in image recognition systems. First of all, analyzing the dependency of data handled in convolution operations, we propose energy-efficient dataflow of convolution and its hardware architecture to minimize energy consumption by maximizing data reuse rate. Previous convolution accelerators, which were mainly focused on high data processing speed, considered limited information on data reuse, resulting in large redundant on-chip memory accesses. To minimize the memory access overhead, we reschedule the dataflow of convolution operations so that the data loaded from on-chip memories are utilized as maximally as possible. As a result, the overall energy consumption is reduced by a factor of 5.9 compared to the previous convolution accelerator. For the inference process, a programmable CNN processor called master-slave core is newly proposed to achieve high flexibility, processing speed and energy efficiency. The heterogeneous instruction set architecture of the master-slave core maximizes computation speed by overlapping the off-chip data transmission and the CNN operations, and reduces power consumption by performing the convolution incrementally to reuse input and partial-sum data as maximally as possible. An inference system can be configured by connecting multiple DSIPs in a form of either 1D or 2D chain structure in order to enhance computation speed further. Compared to the state-of-the-art accelerator, the proposed system enhances the energy efficiency by 2.17x.