Disentangling Content from Style to Overcome Shortcut Learning: A Hybrid Generative-Discriminative Learning Framework

Despite the remarkable success of Self-Supervised Learning (SSL), its generalization is fundamentally hindered by Shortcut Learning, where models exploit superficial features like texture instead of intrinsic structure. We experimentally verify this flaw within the generative paradigm (e.g., MAE) and argue it is a systemic issue also affecting discriminative methods, identifying it as the root cause of their failure on unseen domains. While existing methods often tackle this at a surface level by aligning or separating domain-specific features, they fail to alter the underlying learning mechanism that fosters shortcut dependency. To address this at its core, we propose HyGDL (Hybrid Generative-Discriminative Learning Framework), a hybrid framework that achieves explicit content-style disentanglement. Our approach is guided by the Invariance Pre-training Principle: forcing a model to learn an invariant essence by systematically varying a bias (e.g., style) at the input while keeping the supervision signal constant. HyGDL operates on a single encoder and analytically defines style as the component of a representation that is orthogonal to its style-invariant content, derived via vector projection. This is operationalized through a synergistic design: (1) a self-distillation objective learns a stable, style-invariant content direction; (2) an analytical projection then decomposes the representation into orthogonal content and style vectors; and (3) a style-conditioned reconstruction objective uses these vectors to restore the image, providing end-to-end supervision. Unlike prior methods that rely on implicit heuristics, this principled disentanglement allows HyGDL to learn truly robust representations, demonstrating superior performance on benchmarks designed to diagnose shortcut learning. Self-Supervised Learning (SSL) has recently emerged as a dominant paradigm in representation learning (Grill et al., 2020; Chen et al., 2020a; Sim eoni et al., 2025; Gui et al., 2024; He et al., 2020). Consequently, a significant body of research has aimed to enhance its domain generalization, often by addressing the model's well-documented texture bias (Geirhos et al., 2019). We argue, however, that such approaches often treat the symptom rather than the cause. In this work, we posit that poor generalization stems from a more fundamental problem: the inherent tendency of models towards Shortcut Learning (Geirhos et al., 2020), wherein they exploit superficial features (e.g., texture) that are spuriously correlated with the learning objective, instead of learning the intrinsic, generalizable structure of the data.