Statistical Learning
Scalable Hyperparameter Transfer Learning
Valerio Perrone, Rodolphe Jenatton, Matthias W. Seeger, Cedric Archambeau
In this work, we aim to warm-start BO in the context of hyperparameter optimization (HPO).
Greedy Hash: Towards Fast Optimization for Accurate Hash Coding in CNN
Shupeng Su, Chao Zhang, Kai Han, Yonghong Tian
Despite the considerable progress, it's still a difficult task to realize the real end-to-end training of
The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization
Constantinos Daskalakis, Ioannis Panageas
When they converge, do they converge to local min-max solutions? We characterize the limit points of two basic first order methods, namely Gradient Descent/Ascent (GDA) and Optimistic Gradient Descent Ascent (OGDA).
On Markov Chain Gradient Descent
Tao Sun, Yuejiao Sun, Wotao Yin
Neural Information Processing Systems http://nips.cc/
Stochastic Nested Variance Reduced Gradient Descent for Nonconvex Optimization
Dongruo Zhou, Pan Xu, Quanquan Gu
In this work, we mainly focus on first-order algorithms, which only need the function value and gradient evaluations.
Semi-Supervised Learning with Declaratively Specified Entropy Constraints
Haitian Sun, William W. Cohen, Lidong Bing
Neural Information Processing Systems http://nips.cc/
Stein Variational Gradient Descent as Moment Matching
Stein equation, which may motivate more efficient algorithms.
Multi-Class Learning: From Theory to Algorithm
Jian Li, Yong Liu, Rong Yin, Hua Zhang, Lizhong Ding, Weiping Wang
The theoretical analysis motivates us to devise two effective multi-class kernel learning algorithms with statistical guarantees.
Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes
Loucas Pillaud-Vivien, Alessandro Rudi, Francis Bach
Neural Information Processing Systems http://nips.cc/
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, Michael W. Mahoney
Extensive experiments on multiple networks show that saddle-points are not the cause for generalization gap of large batch size training, and the results consistently show that large batch converges to points with noticeably higher Hessian spectrum.