Feature Matching Intervention: Leveraging Observational Data for Causal Representation Learning

Causal representation learning [Schölkopf et al., 2021] aims to uncover causal features from observations of high-dimensional data, and is emerging as a prominent field at the intersection of deep learning and causal inference. Unlike traditional causal effect of a specific treatment variable, causal representation learning does not treat any observed variable as a potential causal parent. Instead, it focuses on transforming the observational space into a low-dimensional space to identify causal parents. However, despite its promise, recent years have witnessed notable shortcomings in effectively capturing causal features, particularly evident in tasks such as image classification. Numerous experiments over the past decade [Geirhos et al., 2020, Pezeshki et al., 2021, Beery et al., 2018, Nagarajan et al., 2020] have highlighted the failure of models to discern essential features, resulting in a phenomenon where models optimized on training data exhibit catastrophic performance when tested on unseen environments. This failure stems from the reliance of models on spurious features within the data, such as background color in images, rather than the genuine features essential for accurate classification, such as the inherent properties of objects depicted in the images. Consequently, models are susceptible to errors, particularly when faced with adversarial examples. The phenomenon described above is commonly known as out-of-distribution (OOD), with efforts to mitigate it termed as out-of-distribution generalization or domain generalization. To tackle this challenge, many approaches have been proposed.