Sensorformer: Cross-patch attention with global-patch compression is effective for high-dimensional multivariate time series forecasting

However, in the early exploration of multivariate time series forecasting tasks, the Transformer did not demonstrate significant superiority[7]. Nevertheless, Nie et al.[8] soon revealed that a major reason for this issue lies in the single-point token construction method adopted by most approaches (as shown in Figure 1(b1)), which struggles to represent key temporal features such as trends and distributions within a single token. To address this problem, PatchTST was proposed in [8], which significantly improved the performance of the Transformer in multivariate time series forecasting through the use of patch tokens and a channelindependent (CI) forward propagation strategy (as shown in Figure 1(b3)). Subsequently, other patchbased multivariate time series forecasting Transformers, such as Crossformer[9] and TimeXer[10], have also achieved competitive performance. However, according to common assumptions in previous studies, the explicit extraction of crossvariable dependencies is crucial for multivariate time series modeling, as there are often correlations or causal relationships between variables. Research [11] and [12] conducted a more detailed comparison between CI and channel-dependent strategies(CD), and the results revealed that, on almost all deep neural network backbones, including Transformers, the generalization ability of methods based on CI strategy significantly outperforms most CD-based methods. The conclusions of these studies suggest that the CI strategy should become the primary approach for multivariate time series forecasting. However, iTransformer[13], a method that treats each variable sequence as a token and only explicitly extracts cross-variable dependencies, has outperformed PatchTST on many mainstream datasets[10][13][14], achieving SOTA performance.