Goto

Collaborating Authors

 linear classifier


Large Margin Discriminant Dimensionality Reduction in Prediction Space

Neural Information Processing Systems

In this paper we establish a duality between boosting and SVM, and use this to derive a novel discriminant dimensionality reduction algorithm. In particular, using the multiclass formulation of boosting and SVM we note that both use a combination of mapping and linear classification to maximize the multiclass margin. In SVM this is implemented using a pre-defined mapping (induced by the kernel) and optimizing the linear classifiers. In boosting the linear classifiers are pre-defined and the mapping (predictor) is learned through combination of weak learners. We argue that the intermediate mapping, e.g.









SGD on Neural Networks Learns Functions of Increasing Complexity

Dimitris Kalimeris, Gal Kaplun, Preetum Nakkiran, Benjamin Edelman, Tristan Yang, Boaz Barak, Haofeng Zhang

Neural Information Processing Systems

Neural networks have been extremely successful in modern machine learning, achieving the state-of-the-art inawiderangeofdomains, including image-recognition, speech-recognition, andgame-playing [ 14, 18, 23, 37]. Practitioners often train deep neural networks with hundreds of layers and millions of parameters and manage to find networks with good out-of-sample performance.However, this practical prowess isaccompanied by feeble theoreticalunderstanding.