(A) descriptor generation processor for low-power object recognition in video frames

Abstract

In recent years, object recognition plays a key role in mobile vision applications, such as head-mounted displays (HMDs), robot vision, and mini-UAVs. Several types of object recognition processors have been reported, and they have shown real-time 30 frames per second (fps) object recognition with power consumption of 260 ~ 350 mW. Even though their power consumption is reasonable for non-mobile applications or non-power hungry applications, such as surveillance systems and driver assistance systems, in mobile ap-plications, due to the limited battery capacity, the power consumption of object recognition should be reduced further. In the recent object recognition processor, a descriptor generation processor consumes almost half of the total recognition power. Two main bottlenecks of descriptor generation processors are (1) a heavy work-load from real-time 30 fps HD-resolution input and (2) a large number of non-linear operations in descriptor generation process. To solve each bottleneck, a new object recognition flow with a feature reuse model and an LUT-based descriptor generation processor are proposed respectively. In the new object recognition flow with the feature reuse model, inter-frame similarity in video frames is used to reduce the workload of descriptor generation processor. With the proposed flow, over 50% of keypoints of the current frame reuse features of the previous frame. The LUT-based descriptor generation processor has two main features to generate descriptors efficiently. High cost non-linear operations are replaced to LUT-based operations with no accuracy reduction. And a highly utilized keypoint-level pipeline and a pixel-level pipeline are used to increase energy efficiency. The proposed descriptor generation processor with the feature reuse object recognition flow is implemented with a 65nm CMOS technology. The feature reuse engine is operating at 100MHz under 1.2V supply voltage and the descriptor generation processor is operating at 50MHz under 1.2V supply voltage. As a result, it shows 9mW power consumption in average and 385K descriptor/s throughput without the feature reuse scheme. Comparing to the state-of-the art vision processor, 22.9 times higher energy efficiency and 3.5 times higher area efficiency are achieved.