Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram

Electrocardiograms (ECG) are widely employed as a diagnostic tool for monitoring electrical signals originating from a heart. Recent machine learning research efforts have focused on the application of screening various diseases using ECG signals. However, adapting to the application of screening disease is challenging in that labeled ECG data are limited. Achieving general representation through self-supervised learning (SSL) is a well-known approach to overcome the scarcity of labeled data; however, a naive application of SSL to ECG data, without considering the spatial-temporal relationships inherent in ECG signals, may yield suboptimal results. In this paper, we introduce ST-MEM (Spatio-Temporal Masked Electrocardiogram Modeling), designed to learn spatiotemporal features by reconstructing masked 12-lead ECG data. ST-MEM outperforms other SSL baseline methods in various experimental settings for arrhythmia classification tasks. Moreover, we demonstrate that ST-MEM is adaptable to various lead combinations. Through quantitative and qualitative analysis, we show a spatio-temporal relationship within ECG data. Our code is available at https://github.com/bakqui/ST-MEM. The electrocardiogram (ECG) is a non-invasive heart measurement to monitor the electrical activity over time and diagnose diseases. Several supervised learning models have been developed to detect various heart diseases through ECG (Siontis et al., 2021). However, since the types of heart disease are diverse and the experienced cardiologists who can provide labels are limited, learning the ECG representation for each application (i.e., detecting various heart diseases) is challenging. Recently, self-supervised learning (SSL) for general representation has emerged in natural language processing (Kenton & Toutanova, 2019; Brown et al., 2020) and computer vision (Chen et al., 2020; Grill et al., 2020; He et al., 2020; Caron et al., 2021) since it can be leveraged for numerous tasks, such as translation, sentence classification, image classification, and image generation. In ECG-based diagnosis, there were also similar efforts to learn general representation through SSL to overcome the limited resources and detect various heart diseases. ECG-based representation learning through SSL is usually considered in two different learning methods: contrastive and generative learning (Jing & Tian, 2020). Contrastive learning (Sarkar & Etemad, 2020; Le et al., 2023; Gopal et al., 2021; Soltanieh et al., 2022; Kiyasseh et al., 2021; Wei et al., 2022) is a method to ensure similarity in the context before and after data augmentation.