The Evolving Nature of Latent Spaces: From GANs to Diffusion

They have, however, created a specialization of representations. The information bottleneck of previous models proved too restrictive to achieve the desired expressiveness, leaving room for even more distributed, complex, and iterative representations. We could not find any layer representation in these models that would entirely be responsible for the output, leaving behind a synthesis in a strict sense, but creating a'synthesis in a broad sense', where the unity of representation lies in the distributed contribution of the several specialized representations in an emergent synthesis. The Kantian definition of synthetic levels, as mentioned in Fazi, resonates with the specialization of representations found within these models. As Fazi says: "He described 1) the synthesis of apprehension in the intuition, concerning elemental, raw perceptual inputs and producing temporal and spatial structures; 2) the synthesis of reproduction in the imagination, a transcendental act of the mind that unifies apprehended representations with other apprehended representations and corresponds to imagination itself; and 3) the synthesis of recognition in the concept, which concerns understanding and the application of the categories via acts of apperception. What is apprehended in intuition, reproduced in imagination, and recognized in concepts are representations (i.e., intuitions, images, and concepts are representational)."