Learning contextual representations of humans’ trajectory data

Abstract

A trajectory is a sequence of locations and timestamps of a pedestrian, and it has the properties of spatiotemporal dynamics. This indicates that every location has different characteristics depending on which trajectory it was on and which time it appeared. The trajectory also reflects personal characteristics such as age. In previous studies, however, the location was statically assigned with one embedding vector without considering these dynamic of locations. Therefore, this study proposes Geo-BERT, a dynamic geo-embedding model that reflects the spatio-temporal dynamics of locations using BERT architecture in the field of natural language processing. In the pre-training, masked location completion task was performed, which predicts randomly masked locations in the trajectory so that the model could learn the context of trajectories. In additions, instead of next sentence prediction task of BERT, our model learned the purpose of the trajectory by predicting the location type (e.g., business district) of the last location in the trajectory. The two tasks were performed together in the pre-training stage. After pretraining, various downstream tasks can be performed through fine-tuning process. This study sets two downstream tasks: (1) future visitor prediction, and (2) transportation mode classification. Our model showed competitive performance in this downstream task compared to other baseline models.