Deep Learning
Redundant representations help generalization in wide neural networks
Deep neural networks (DNNs) defy the classical bias-variance trade-off: adding parameters to a DNN that interpolates its training data will typically improve its generalization performance. Explaining the mechanism behind this "benign overfitting" in deep networks remains an outstanding challenge.
BulletTrain: Accelerating Robust Neural Network Training via Boundary Example Mining Weizhe Hua
Neural network robustness has become a central topic in machine learning in recent years. Most training algorithms that improve the model's robustness to
Finite-Time Analysis of Adaptive Temporal Difference Learning with Deep Neural Networks
However, from the theoretical perspective, establishing theoretical convergence guarantees for training DNNs is much more complicated than that for the linear approximation algorithms, which is still widely open.
Linear Dynamical Systems as a Core Computational Primitive
Furthermore, LDStack is amenable to linear systems theory.
Surprising Instabilities in Training Deep Networks and a Theoretical Analysis Y uxin Sun 1 Dong Lao
However, the practice has out-paced the theory.
Maximum Class Separation as Inductive Bias in One Matrix
The main observation behind our approach is that separation does not require optimization but can be solved in closed-form prior to training and plugged into a network.