Thermal image-based baby and adult detection robust to residual thermal heat marks

Abstract

Unintentional injuries in babies still frequently occur at households even today, and many of these injuries occur outside of baby cribs. Nevertheless, many conventional smart baby monitoring systems utilize facial image based baby detection methods, which places a limit for usage in diverse household environments. In addition, with recent concerns raised for the invasion of privacy for in-home baby monitoring systems, the privacy of the users need to be preserved. To overcome these limitations, thermal image-based full-body images of babies and adults were collected for detection of babies and adults

however, when using thermal images for detection tasks, misdetections due to thermal residual heat marks could incur, which could lead to dangerous consequences for babies in baby monitoring perspectives. In fact, from a preliminary experiment on thermal-image based baby and adult detection with RetinaNet, thermal residual heat mark accounted for the second most frequent source of misdetection in babies. Therefore, in this dissertation, a sequential thermal-image based adult and baby detection is presented that is robust to thermal residual heat marks. The proposed method leveraged thermal temperature difference between sequential image frame inputs as well as the relevant spatiotemporal architectures that rescales features not only based on global context of thermal images, but also thermal temperature difference characteristics. Results on the collected thermal image adult and baby detection dataset indicated that the proposed method showed best robustness in terms of misdetection due to thermal residual heat marks.