Interpretation of microbolometer mechanisms for gyro-aided blur kernel rendering and its applications

Abstract

Various types of motion blur are frequently observed in the images captured by sensors based on thermal and photon detectors. The difference in mechanisms between thermal and photon detectors directly results in different patterns of motion blur. Motivated by this observation, we propose a novel method to synthesize blurry images from sharp images by analyzing the mechanisms of the thermal detector. Further, we propose a novel blur kernel rendering method, which combines our proposed motion blur model with the inertial sensor in the thermal image domain. The accuracy of the blur kernel rendering method is evaluated by the task of thermal image deblurring. We construct a synthetic blurry image dataset based on acquired thermal images using an infrared camera for evaluation. This dataset is the first blurry thermal image dataset with ground-truth images in the thermal image domain. Qualitative and quantitative experiments are extensively carried out on our dataset, which shows that our proposed method outperforms state-of-the-art methods. Furthermore, we applied the gyro-aided blur kernel rendering method to the object tracking. We propose a novel gyro-based tracking assistant designed to excel in aerial environments, where the use of drones equipped with infrared cameras is expanding rapidly. Our proposed method comprises two sub-modules. First, the search region prediction module independently estimates the search region position for the current frame using only gyroscope sensor data. The prediction module estimates the displacement of the search area location between adjacent frames due to camera motion using the homography transform. Second, the search region deblurring module renders a blur kernel using only gyroscope sensor data. The deblurring module introduces an approach that models an infrared sensor mechanism and merges this model with the homography transform to synthesize the blur kernel. The rendered blur kernel is used to deblur the search region with a deconvolution algorithm. To quantitatively evaluate our proposed method, we constructed a dataset ourselves. We collected and synchronized gyroscope sensor data and infrared images in a configuration similar to a drone environment. Our dataset comprises 15 sequences and two classes, with camera motion effects encompassing six distinct steps. Our experiments were structured into two main categories. First, we analyzed the degradation in tracking performance caused by camera motion. This analysis revealed that displacement has a more significant impact on tracking performance than motion blur. Second, we evaluated the effectiveness of our gyro-based tracking assistant. Through extensive quantitative experiments, we demonstrated that our integrated tracker outperforms the use of state-of-the-art trackers alone.