Deep Learning
Directional convergence and alignment in deep learning
The above theories, with finite width networks, usually require the weights to stay close to initialization in certain norms. By contrast, practitioners run their optimization methods as long as their computational budget allows [Shallue et al., 2018], and if the data can be perfectly classified, the
The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks
In this work, we show that these common perceptions can be completely false in the early phase of learning.
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms
Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade.
Contrastive Learning with Adversarial Examples
Deep networks have enabled significant advances in many machine learning tasks over the last decade. However, this usually requires supervised learning, based on large and carefully curated datasets.
