PeriodNet: Boosting the Potential of Attention Mechanism for Time Series Forecasting

PeriodNet: Boosting the Potential of Attention Mechanism for Time Series Forecasting Bowen Zhao, Huanlai Xing, Zhiwen Xiao, Jincheng Peng, Li Feng, Xinhan Wang, Rong Qu, Hui Li The proposed PeriodNet hybridizes a period attention mechanism, an iterative grouping mechanism, and a period diffuser architecture to achieve accurate multivariate time series forecasting. The period attention mechanism captures temporal similarities among adjacent periods to improve time series modeling. The period diffuser architecture leverages multi-scale period features extracted by the encoder to enhance the accuracy and efficiency of time series forecasting. Abstract The attention mechanism has demonstrated remarkable potential in sequence modeling, exemplified by its successful application in natural language processing with models such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT). Despite these advancements, its utilization in time series forecasting (TSF) has yet to meet expectations. Exploring a better network structure for attention in TSF holds immense significance across various domains. In this paper, we present PeriodNet with a brand new structure to forecast univari-ate and multivariate time series. PeriodNet incorporates period attention and sparse period attention mechanism for analyzing adjacent periods. It enhances the mining of local characteristics, periodic patterns, and global dependencies. For efficient cross-variable modeling, we introduce an iterative grouping mechanism which can directly reduce the cross-variable redundancy. To fully leverage the extracted features on the encoder side, we redesign the entire architecture of the vanilla Transformer and propose a period diffuser for precise multi-period prediction. Through comprehensive experiments conducted on eight datasets, we demonstrate that PeriodNet outperforms six state-of-the-art models in both univariate and multivariate TSF scenarios in terms of mean square error and mean absolute error.