Towards diverse perspective learning with select over multiple temporal poolings

Abstract

In Time Series Classification (TSC), in order to address the issue of losing temporal information caused by global pooling, temporal pooling methods that consider sequential information have been proposed. However, we found that each temporal pooling has a distinct mechanism, and can perform better depending on time series data. In this paper, we propose a novel temporal pooling method with diverse perspective learning: Select over Multiple Temporal Pooling (SoM-TP). SoM-TP dynamically selects the optimal temporal pooling among multiple methods for each data by attention. The dynamic pooling selection is motivated by the ensemble concept of Multiple Choice Learning (MCL) which selects the best among multiple outputs. To achieve non-iterative optimization, we define a perspective loss. The loss works as a regularizer to reflect all the pooling perspectives. Our massive case study using Layer-wise Relevance Propagation (LRP) reveals the limitation of a single perspective that each temporal pooling has, and ultimately demonstrates the necessity of diverse perspectives achieved by SoM-TP. Extensive experiments are done with the UCR/UEA repository and large datasets.