(A) low-power high-performance DNN processor for mobile platforms

Abstract

Recently, deep learning, realized with deep neural networks (DNNs), has become universal in all-around applications due to its overwhelming performance and universal applicability. It has changed the paradigm of machine learning and brought significant progress in vision, speech, language processing, and many other applications. In 2012, a DNN named AlexNet dramatically reduced the image classification error by 10% for the ImageNet dataset. It has been developed rapidly and now performs human level accuracy in image classification. Due to rapid development, more and more companies are providing services that utilize deep learning. These services are mostly delivered in the form of cloud computing via servers in the data center due to the large computational requirements of the deep learning algorithm. In cloud computing, tasks requested by users are sent to the server through the Internet, and then the results are sent back to the user after the operation is completed on the server. In contrast, edge computing processes the tasks within the edge device itself. These edge computing benefits over cloud computing in terms of latency, data transfer costs, security, stability, and reliability, enabling better services. In particular, for video-based real-time services, these features become more prominent. The goal of this research is to implement intelligence-on-things or smart machines by enabling deep learning through edge computing on mobile platforms such as smartphones, drones, IoT devices, wearable devices, and robots. However, current mobile computing units could not meet the throughput and power requirements to process deep learning in real-time. Therefore, implementing a deep learning ASIC with higher energy efficiency is essential and is presented in this dissertation.

In this dissertation, needs, validity, and characteristics of deep learning ASIC are introduced with key schemes for higher energy efficiency. Key schemes include existing and newly proposed ones and are divided into reduced numerical precision, processing and data flow optimization, utilization of reusability, sparsity exploit, and customized ALU. Under these bases, an energy-efficient deep learning ASIC named DNPU is presented with the real chip implementation. It has the following key features:

1) A reconfigurable heterogeneous architecture to support various DNNS 2) An off-chip access-efficient workload division method to handle large data with a limited on-chip memory 3) On-line self-tuning layer-by-layer dynamic-fixed-point to reduce the bit-width of activations 4) A quantization table-based matrix multiplication to reduce off-chip memory accesses and remove duplicated multiplications. 5) A computational SRAM-based stereo matching processor for RGB-depth 4-ch support