Multimodal time series (MTS) anomaly detection is crucial for maintaining the safety and stability of working devices (e.g., water treatment system and spacecraft), whose data are characterized by multivariate time series with diverse modalities. Although recent deep learning methods show great potential in anomaly detection, they do not explicitly capture spatial-temporal relationships between univariate time series of different modalities, resulting in more false negatives and false positives. In this paper, we propose a multimodal spatial-temporal graph attention network (MST-GAT) to tackle this problem. MST-GAT first employs a multimodal graph attention network (M-GAT) and a temporal convolution network to capture the spatial-temporal correlation in multimodal time series. Specifically, M-GAT uses a multi-head attention module and two relational attention modules (i.e., intra- and inter-modal attention) to model modal correlations explicitly. Furthermore, MST-GAT optimizes the reconstruction and prediction modules simultaneously. Experimental results on four multimodal benchmarks demonstrate that MST-GAT outperforms the state-of-the-art baselines. Further analysis indicates that MST-GAT strengthens the interpretability of detected anomalies by locating the most anomalous univariate time series.

Structural health monitoring (SHM) has emerged as a vital field of research, geared towards preserving the longevity and safety of civil infrastructure [1]. A critical component of SHM is the analysis of vibration time series data, which offers insights into the behavior, health, and performance of structures [2]. As infrastructure, especially in urban regions, is subject to a myriad of dynamic forces--ranging from wind to traffic loads - it becomes pivotal to extract clear and meaningful data from the complex vibration signatures that these forces induce. However, one of the significant challenges plaguing SHM practitioners is the interference of noise in these vibration signals, which can distort interpretations and lead to unreliable conclusions. The dynamic response of structures is often manifested as multimodal vibrations, meaning multiple modes or patterns of vibration coexist. These modes, each characterized by its frequency and shape, provide a fingerprint of the structure's health and dynamic properties.