DynaLAP: Human Activity Recognition in Fixed Protocols via Semi-Supervised Variational Recurrent Neural Networks With Dynamic Priors

Abstract

Learning the route and order of tasks can be critical to human activity recognition (HAR) for fixed protocols of movement. In this article, we propose a novel framework, DynaLAP, a semi-supervised variational recurrent neural network (VRNN) with a dynamic prior distribution, to perform activity recognition in fixed protocols. DynaLAP takes single tri-axial accelerometry data as input and causally classifies the activity of 10-30-s windows at a time. DynaLAP learns not only a window-specific short-term state, but also a long-term dynamic state iteratively updated throughout the protocol's measurements. Additionally, instead of using a stationary prior distribution of activity classes, DynaLAP learns a dynamic prior that updates for each window. DynaLAP thereby learns protocol-specific dynamics when trained on data from subjects abiding by a fixed protocol. Two datasets from previously published literature were used to evaluate DynaLAP: the fully labeled MotionSense dataset of 24 subjects and a weakly labeled dataset of 17 subjects collected at the Georgia Institute of Technology. For each dataset, we varied the number of training labels used from a single subject's data to the entire dataset. DynaLAP outperformed previous supervised and semi-supervised HAR approaches by 6-42 percentage points, with F1 scores that remained above 80%. These results suggest that DynaLAP can achieve state-of-the-art HAR performance in fixed protocols by learning protocol-specific dynamics, especially in weakly and scarcely labeled settings. DynaLAP could ultimately reduce the necessity for labor-intensive annotation efforts in HAR applications involving routine activities (e.g., military training).