High energy efficient DNN inference and training processor with sparsity optimization and neuromorphic computing

Abstract

This paper focuses on sparsity utilization in a neural network (DNN) for energy-efficient processing of mobile devices and an optimization method through neuromorphic computing. While previous accelerators have utilized sparsity in DNN to achieve energy efficiency, they have been limited to handling sparsity that occurs in input due to activation functions, and their sparse processing methods are only effective for DNN inference. In this paper, four DNN accelerators (TSUNAMI, SNPU, C-DNN, and Neuro-CIM) are proposed to validate the new sparsity utilization techniques in DNN sequentially. TSUNAMI generates weight sparsity through iterative pruning in addition to input and output sparsity, and achieves consistently high computational speedup during DNN training by skipping computations for two types of high sparsity among input/weight/output. SNPU increases input sparsity by performing spike encoding for inputs of DNN and integrates newly proposed spike encoders to reduce spike frequency, enabling low-power always-on applications. C-DNN allocates inputs with low spike frequency to neuromorphic computing and inputs with high spike frequency, which can increase power consumption, to conventional deep neural network computing, achieving high energy efficiency. Neuro-CIM implements the such as accumulation and firing logic of neuromorphic computing as in-memory operations, and achieves low power consumption by designing without analog-digital converters, which consume significant power in previous in-memory computing structures.