ifm

In this section we give proofs for all the results in Sec. 2, which explores the phenomenon of feature suppression in contrastive learning using the InfoNCE loss. We invite the reader to consult Sec. Recall, for a measure on a space U and a measurable map h: U! V let h# denote the pushforward h# (V)= (h We now recall the definition of feature suppression and distinction. Consider an encoder f: X! S However here our goal is to establish that it is possible for InfoNCE optimal encoders both to suppress features in the sense of Def. 1, but also to separate concepts out in a desirable manner. For this purpose we found this strong notion of distinguishing to suffice. Since g is injective, we know there exists a left inverse h: X!Zsuch that h g(z) =z for all z 2Z.