DSTP-RNN: a dual-stage two-phase attention-based recurrent neural networks for long-term and multivariate time series prediction

Yeqi Liu, Chuanyang Gong, Ling Yang, Yingyi Chen * Abstract Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is to capture the spatial correlations at the same time, the spatiotemporal relationships at different times and the long-term dependence of the temporal relationships between different series. Attention-based recurrent neural networks (RNN) can effectively represent the dynamic spatiotemporal relationships between exogenous series and target series, but it only performs well in one-step time prediction and short-term time prediction. The first phase produces violent but decentralized response weight, while the second phase leads to stationary and concentrated response weight. Secondly, we employ multiple attentions on target series to boost the long-term dependence. Finally, we study the performance of deep spatial attention mechanism and provide experiment and interpretation. Our methods outperform nine baseline methods on four datasets in the fields of energy, finance, environment and medicine, respectively. Keywords: Time series prediction; Spatiotemporal relationship; Attention mechanism; Dual-stage two-phase model; Deep attention network. 1 Introduction Recent developments in the Internet of Things and Big Data have led to the continuous expansion of data scale(Le & Ge, 2019). Hence, the long-term prediction of multivariate time series has more practical significance, e.g., it is more significant to forecast the weather of one or more days than to forecast the weather of the next hour in the future. However, the long-term prediction of multivariate time series is still a challenging problem, which is mainly reflected in the feature representation and selection mechanism of spatiotemporal relationships between different series.