loss surface
SHOT: Suppressing the Hessian along the Optimization Trajectory for Gradient-Based Meta-Learning
To validate the effectiveness of SHOT, we conduct empirical tests on standard few-shot learning tasks and qualitatively analyze its dynamics.
- North America > United States > California (0.04)
- Europe > Germany > North Rhine-Westphalia > Upper Bavaria > Munich (0.04)
- Asia > South Korea > Seoul > Seoul (0.04)
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies Oscar Li1, James Harrison, Jascha Sohl-Dickstein, Virginia Smith
First-order optimization methods are a foundational tool in machine learning.
- North America > United States > Virginia (0.40)
- North America > United States > Pennsylvania > Allegheny County > Pittsburgh (0.04)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (0.93)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Optimization (0.87)
- Information Technology > Artificial Intelligence > Machine Learning > Evolutionary Systems (0.66)
A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training
However, the training of a sparse DNN encounters great challenges in achieving optimal generalization ability despite the efforts from the state-of-the-art sparse training methodologies. To unravel the mysterious reason behind the difficulty of sparse training, we connect network sparsity with the structure of neural loss functions and identify that the cause of such difficulty lies in a chaotic loss surface.
- North America > United States (0.46)
- Asia > China > Tianjin Province > Tianjin (0.04)
- Europe > Italy (0.04)
Hybrid 8-bit Floating Point (HFP8) Training and Inference for Deep Neural Networks
Xiao Sun, Jungwook Choi, Chia-Yu Chen, Naigang Wang, Swagath Venkataramani, Vijayalakshmi (Viji) Srinivasan, Xiaodong Cui, Wei Zhang, Kailash Gopalakrishnan
Neural Information Processing Systems http://nips.cc/
- North America > United States (0.04)
- North America > Canada (0.04)
- Asia > South Korea (0.04)
- Asia > Middle East > Jordan (0.04)
Large Scale Structure of Neural Network Loss Landscapes
Stanislav Fort, Stanislaw Jastrzebski
Neural Information Processing Systems http://nips.cc/
- Europe > Switzerland > Zürich > Zürich (0.14)
- North America > United States (0.04)
- North America > Canada (0.04)
- (2 more...)
Pre-TrainYourLoss: EasyBayesianTransfer LearningwithInformativePriors
However,transfer learningis quickly becoming mainstream practice in deep learning.
Asymmetric Valleys: Beyond Sharp and Flat Local Minima
Haowei He, Gao Huang, Yang Yuan
Neural Information Processing Systems http://nips.cc/
- Asia > Middle East > Jordan (0.04)
- Asia > China > Beijing > Beijing (0.04)
- North America > Canada (0.04)
- North America > Canada (0.04)
- Asia > Middle East > Jordan (0.04)
- Asia > India > Karnataka > Bengaluru (0.04)