PFformer: A Position-Free Transformer Variant for Extreme-Adaptive Multivariate Time Series Forecasting

PFformer: A Position-Free Transformer V ariant for Extreme-Adaptive Multivariate Time Series Forecasting Yanhong Li 1 and David C. Anastasiu 1 1 Department of Computer Science and Engineering, Santa Clara University, Santa Clara, CA, USA Multivariate time series (MTS) forecasting is vital in fields like weather, energy, and finance. However, despite deep learning advancements, traditional Transformer-based models often diminish the effect of crucial inter-variable relationships by singular token embedding and struggle to effectively capture complex dependencies among variables, especially in datasets with rare or extreme events. These events create significant imbalances and lead to high skewness, complicating accurate prediction efforts. This study introduces PFformer, a position-free Transformer-based model designed for single-target MTS forecasting, specifically for challenging datasets characterized by extreme variability. EFE effectively encodes inter-variable dependencies by mapping related sequence subsets to high-dimensional spaces without positional constraints, enhancing the encoder's functionality. PFformer shows superior forecasting accuracy without the traditional limitations of positional encoding in MTS modeling. We evaluated PFformer across four challenging datasets, focusing on two key forecasting scenarios: long sequence prediction for 3 days ahead and rolling predictions every four hours to reflect real-time decision-making processes in water management. PF-former demonstrated remarkable improvements, from 20% to 60%, compared with state-of-the-art models.