Symmetries are prevalent in deep learning and can significantly influence the learning dynamics of neural networks. In this paper, we examine how exponential symmetries - a broad subclass of continuous symmetries present in the model architecture or loss function - interplay with stochastic gradient descent (SGD). We first prove that gradient noise creates a systematic motion (a "Noether flow") of the parameters θ along the degenerate direction to a unique initialization-independent fixed point θ

In large-scale applications, datasets often contain billions of high-dimensional points. Grouping similar data points into clusters is crucial for understanding and organizing datasets.

Empirically, we verify the computational efficiency of our methods.

In many search applications related to passage retrieval, text entailment, and sub-graph search, the query and each'document' is a set of elements, with a document

TBAL is emerging as a widely-used solution in practice.