Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame

Specifically, we introduce the notion of nearest simplex ETF geometry for the penultimate layer features at any given training iteration, by formulating it as a Riemannian optimisation.