Improving Self-Supervised Learning by Characterizing Idealized Representations

Our goal is to provide a simple conceptual framework to think about those questions. To derive such a framework, we ask ourselves: what are the ideal requirements that ISSL representations should aim to satisfy? We prove necessary and sufficient requirements to ensure that probes from a specified family, e.g.