TiVaT: Joint-Axis Attention for Time Series Forecasting with Lead-Lag Dynamics

Multivariate time series (MTS) forecasting plays a crucial role in various realworld applications, yet simultaneously capturing both temporal and inter-variable dependencies remains a challenge. Conventional Channel-Dependent (CD) models handle these dependencies separately, limiting their ability to model complex interactions such as lead-lag dynamics. To address these limitations, we propose TiVaT (Time-Variable Transformer), a novel architecture that integrates temporal and variate dependencies through its Joint-Axis (JA) attention mechanism. Ti-VaT's ability to capture intricate variate-temporal dependencies, including asynchronous interactions, is further enhanced by the incorporation of Distance-aware Time-Variable (DTV) Sampling, which reduces noise and improves accuracy through a learned 2D map that focuses on key interactions. Notably, it excels in capturing complex patterns within multivariate time series, enabling it to surpass or remain competitive with state-of-the-art methods. This positions TiVaT as a new benchmark in MTS forecasting, particularly in handling datasets characterized by intricate and challenging dependencies. However, handling both temporal and inter-variable dependencies in MTS remains a challenge. MTS models are typically classified as either Channel-Independent (CI) or Channel-Dependent (CD) based on how they handle inter-variable relationships. CI models process variables independently, which makes them resilient to noise and overfitting but neglects crucial inter-variable dependencies required for complex datasets. Recent CD models, such as iTransformer (Liu et al., 2023) and CARD (Wang et al., 2024b), use Transformer architectures to model these dependencies, improving predictive accuracy.