(A) Software/hardware co-design framework for scalable and reconfigurable CNN accelerator

Abstract

Various accelerators are continuously being developed to support a tremendous amount of convolutional neural networks (CNNs) computation. However, as new networks continue to emerge, there is a limit to the practicality of an accelerator dedicated to a specific network. Flexible dataflows and accelerator designs are required to deal with the various data shapes, including channel depth and input feature map dimension. In this aspect, row stationary dataflow is revisited for its flexibility. However, row stationary dataflow has challenges on data mapping latency and complexity of data order.

This paper proposes a software/hardware co-design framework that spans the performance optimization to the hardware architecture. The proposed framework provides the options of architecture that have the best performance considering the resources using latency modeling. The optimal data mapping variables for each layer are automatically generated by latency modeling software and applied to the implemented accelerator. The error rate between the software and hardware for the end-to-end operation result of VGG 16 is only 1.2 % which shows the reliability of the proposed framework. Using the proposed framework, the data mapping latency and data-reordering challenge of the row stationary dataflow, a state-of-art reusable dataflow, is resolved. The proposed framework enables the accelerator with row stationary dataflow to achieve x1.86 higher PE utilization compared to the baseline architecture. The accelerator is implemented on a Xilinx VCU118 board with an XCVU9P FPGA. It achieves 454.5 giga operation per second (GOPS) at 200 MHz. This research is conducted in collaboration with Donghyuk Kim and Gyeongcheol Shin, master's students of the laboratory.