timedrl
TimeDRL: Disentangled Representation Learning for Multivariate Time-Series
Chang, Ching, Chan, Chiao-Tung, Wang, Wei-Yao, Peng, Wen-Chih, Chen, Tien-Fu
Abstract--Multivariate time-series data in numerous realworld applications (e.g., healthcare and industry) are informative but challenging due to the lack of labels and high dimensionality. Recent studies in self-supervised learning have shown their potential in learning rich representations without relying on labels, yet they fall short in learning disentangled embeddings and addressing issues of inductive bias (e.g., transformationinvariance). To tackle these challenges, we propose TimeDRL, a generic multivariate time-series representation learning framework with disentangled dual-level embeddings. The first challenge in SSL for time-series data is learning I. Existing approaches Multivariate time-series data are widely used in various focus on deriving either timestamp-level [12], [13] or instancelevel applications, such as forecasting for electric power [1], [2], embeddings [14]-[16], but not both at the same time. These time-series timestamp-level embeddings are effective for anomaly datasets are rich in information, but the patterns within and detection and forecasting, whereas instance-level embeddings across temporal dimensions are not discernible by humans, are suited for classification and clustering tasks [17]. Recently, we can theoretically avoid explicitly deriving instance-level there has been a growing trend among researchers to first learn embeddings by extracting them from timestamp-level embeddings representations/embeddings from a large amount of unlabeled using pooling methods (as illustrated in Figure 1(a)) [12], data using unsupervised representation learning and then to this approach often results in the anisotropy problem [18]- fine-tune these models with a limited amount of labeled data [20], where the embeddings are confined to a narrow cone for specific downstream tasks. Self-supervised learning (SSL) is a prominent method To the best of our knowledge, how to disentangle within unsupervised representation learning, which captures instance-level embeddings from timestamp-level embeddings generalizable representations from unlabeled data with pretext in the time-series domain remains an unexplored problem. The left sections (a) and (c) represent predictive learning, utilizing a single representation to predict inherent data characteristics.