Robust detection of pedestrian/cyclist by neuromorphic visual processing

Abstract

There have been many researches carried out in both academia and industry about pedestrian detection based on visual processing. The pedestrian or cyclist is exposed to a lot of danger on the street as reflected by statistics collected all over the world. To enhance the safety of the pedestrian or cyclist there have been many breakthroughs in computer vision research and the rapid growth of demand in its applications to actual vehicles. However, the reliability and the robustness of vision algorithm are still very much in doubt to improve the safety of pedestrian or cyclist for any environment. Most computer vision-based algorithms are complex in nature which causes the cost issue for the vehicle manufacturers or the end user, while most of those are for use in day time under overcast weather conditions– as the functionality drops significantly in night time, raining or both conditions. The objective of this work was to capture the reliability of biological eye and the accuracy of computer vision by implementing neuromorphic visual processing, to achieve both the low cost ASIC implementation and the robust functionality at dark or wet day. The neuromorphic visual processing uses the basis of result from Hubel and Wiesel’s experiment on visual cortex. We have developed the unique neuromorphic visual processing which mimics the fundamental function of brain’s visual cortex. And since its neuromorphic implementation, it is possible to implement the pedestrian/cyclist detection device in low cost CMOS technology of either analogue-mixed ASIC or digital ASIC. Our neuromorphic visual processing extracts different directions of orientation features, which is then passed to the neural networks for template processing to determine the detection of human object. The template resembles that of upper torso of human body or featured figure of particular target object. The stereo visual signal was employed as an option to further improve accuracy even in complex background. In this paper the successful result of pedestrian/cyclist detection will be presented based on the video data captured in various places in Korea, in London, in Chinese city. The data sets all vary slightly from each other as the surrounding is very different. The captured data sets include the night time, raining, or both conditions, which required minor changes in processing parameters to meet the change in the image data for the successful detection. There were few successful test runs for the pedestrian/cyclist using monocular sensor during night time or rainy day with success. Experimentation was carried out on the stereo data set and it was possible to detect the depth of the object within the image making it possible to differentiate between foreground and background. This, in turn, will improve the accuracy rate in heavily clustered scenes such as in city or rural areas with dense forest. The detection of cycle or bike at the crossroad was successful by detecting the front wheel. The nenuromorphic pedestrian/cyclist/cycle detection is the single image frame based processing. In conclusion, this research has successfully detected the pedestrian or cyclist (pedestrian detection rate:95%) by using a neuromorphic approach to reduce the cost for implementation whilst improving the robustness and latency. The research showed the feasibility of CMOS implementation of the neuromorphic ASIC, as an ongoing project with Hyundai Motor Co., with the implementation of neuromorphic ASIC that uses the proposed neuromorphic pedestrian detection to the virtual engine sound system for warning pedestrians near to the silent electric vehicle.