In order to address this limitation, we propose a novel class of functions that can characterize the loss landscape of modern deep models without requiring extensive over-parametrization and can also include saddle points.

To evaluate MLLMs' A VR abilities systematically, we introduce

Pruning has emerged as a pivotal compression strategy, introducing sparsity to enhance both memory and computational efficiency.

One of the key challenges in machine learning is to find interpretable representations of learned functions.

In the literature of studying training dynamics of transformers, several simplifications are commonly adopted such as weight reparameter-ization, attention linearization, special initialization, and lazy regime.