Multi-spectral image fusion based on a new joint bilateral filter for video analytics

Abstract

In this paper, we propose a novel human detection system with thermal and visible image sensors using a new joint bilateral filter. There is a limitation to the performance of devices which attempt to detect humans by using only one sensor. We therefore use two sensors: an infrared sensor and a color image sensor. To fuse the best features of both types of sensor in order to achieve superior performance in human detection, we use low-level image fusion. To carry out this image fusion process, we propose a new joint bilateral filter which is useful for fusion of the edge information in the visible image and the white region in the thermal image and segmentation method of thermal image by using the facet filter and determinant of Hessian matrix. This image fusion process effectively removes the regions of human shadow which is trouble of background subtraction. To achieve this effect, we make the background model with multi-spectral fusion results. The results from different pairs of current fused images and the background model are subtracted to detect the human effectively. Experimental results show that effective and easy human detection can be achieved by the proposed algorithm.