Neural differential equation-based positional embedding for irregular time-series forecasting

Abstract

Time series forecasting plays a pivotal role in various domains such as energy management, meteorology, and financial markets. While irregularly sampled time series data, characterized by non-uniform intervals, is prevalent in practical applications, existing research has pursued separate directions for regular and irregular time series forecasting. Regular-time-series-forecasting models predominantly employ transformer architectures, while irregular time series analysis relies on recurrent models like RNN-based models and more recently neural-differential-equation-based models. Given the challenges posed by irregular time series, we propose NCDE-PE, a method leveraging neural controlled differential equations for encoding temporal information. This approach is complemented by a positional embedding technique based on Controlled Differential Equations (CDEs), aiming to bridge the gap between regular and irregular time series forecasting models. Challenges in learning embeddings from irregularity are addressed, considering asynchronous measurements and irregular time gaps. Additionally, the necessity for effective positional embeddings in irregular time series forecasting is explored, emphasizing the learning of recurrent properties. Our contributions include the introduction of NCDE-PE as a novel positional embedding method, effectively representing positional information and learning recurrence to enhance forecasting accuracy. The proposed method outperforms existing techniques across various irregularly-sampled time series datasets, showcasing its enhanced efficacy.