Transformer based early classification for real-time human activity recognition in smart space

Abstract

Human activity recognition (HAR) plays an important role in intelligent systems. Ambient sensors are utilized to avoid privacy concerns and to collect data streams in a less intrusive manner in smart space. In scenarios requiring immediate service provision, the systems must perform HAR in real-time. Since it is difficult to segment the exact transition point in real-time, data unrelated to the target activity can appear at the beginning of the time series, which we call unrefined data. It leads us to a new challenge that the HAR model recognizes a user's activity as early as possible with unrefined data. In this paper, we propose a Transformer-based real-time HAR scheme that filters out unrefined data. The proposed model lowers the weights for unrefined data and passes information related to target activity to the early classifier. The early classifier adaptively determines the timing of activity recognition. We evaluate the proposed model with 3 public datasets collected on testbeds with ambient sensors installed. The results show that the use of the filtering network that filters out unrefined data improves recognition performance.