Stand-alone wearable hybrid vision system for real-time terrain classification with enhanced light robustness

Abstract

Terrain detection is important in human walking because lower extremity control strategies vary depending on the terrain. Therefore, walking assistance devices must proficiently classify the terrain for efficient gait assistance. We employ a stereo depth camera-LiDAR hybrid vision system affixed to the user’s trunk, and a deep learning classifier to create a vision-based terrain classification system to classify level ground, ramps, and stairs. We conducted a comprehensive assessment of classification performance across diverse lighting conditions and terrains where vision sensors may be less accurate, including clear days, overcast days, and nighttime scenarios. Additionally, we evaluated the system’s performance in low-light environments where vision sensors might face challenges. Our comparative analysis encompassed six algorithms, including the Ramer Douglas Peucker algorithm and Convolutional Neural Network(CNN) employed in prior research, as well as PointNet, a conventional point cloud classification method. The proposed classifier exhibited the highest classification accuracy at 92.16% among the six. It achieved an accuracy of 93.73% even in low-light conditions. Furthermore, we present a real-time classification system and illustrate classifying diverse terrains.