A 49.5 mW Multi-Scale Linear Quantized Online Learning Processor for Real-Time Adaptive Object Detection

Abstract

Online training is essential to maintain a high object detection (OD) in various environments. However, additional computation workload, EMA, and high bit precision is the problem of conventional online learning scheme on mobile devices. Therefore, a low power real-time online learning OD processor is proposed with three key features. First, multi-scale linear quantization (MSLQ) and MSLQ-aware PE structure are proposed for low-bit computation. Second, channel-wise gradient skipping is proposed to reduce computation and EMA based on temporal correlation. These schemes reduce similar to 56% of computation burden and similar to 30% of EMA, and also improve detection accuracy. Lastly, gradient norm clipping with norm estimation achieves 3.8 mAP improvement at YouTube-Objects dataset by fast adaptation with under 1% of the additional computation. Finally, the proposed online learning OD processor is implemented in 28 nm CMOS technology and occupies 1.2 mm(2). The proposed processor achieves 78 mAP of detection accuracy at the YouTube-Objects dataset. Compared to the previous OD processor, this brief shows state-of-the-art performance by achieving 49.5 mW power consumption and 34.4 frame-per-second real-time online learning OD on mobile devices.