An Energy-Efficient Sparse Deep-Neural-Network Learning Accelerator with Fine-Grained Mixed Precision of FP8–FP16

Abstract

Recently, several hardware have been reported for deep-neural-network (DNN) acceleration, however, they focused on only inference rather than DNN learning that is crucial ingredient for user adaptation at the edge-device as well as transfer learning with domain-specific data. However, DNN learning requires much heavier floating-point (FP) computation and memory access than DNN inference, thus, dedicated DNN learning hardware is essential. In this letter, we present an energy-efficient DNN learning accelerator core supporting CNN and FC learning as well as inference with following three key features: 1) fine-grained mixed precision (FGMP); 2) compressed sparse DNN learning/inference; and 3) input load balancer. As a result, energy efficiency is improved 1.76× compared to sparse FP16 operation without any degradation of learning accuracy. The energy efficiency is 4.9× higher than NVIDIA V100 GPU and its normalized peak performance is 3.47× higher than previous DNN learning processor. © 2018 IEEE.