We investigate a reduction of supervised learning to game playing that reveals new connections and learning methods. For convex one-layer problems, we demonstrate an equivalence between global minimizers of the training problem and Nash equilibria in a simple game. We then show how the game can be extended to general acyclic neural networks with differentiable convex gates, establishing a bijection between the Nash equilibria and critical (or KKT) points of the deep learning problem. Based on these connections we investigate alternative learning methods, and find that regret matching can achieve competitive training performance while producing sparser models than current deep learning approaches.

Neural Information Processing Systems http://nips.cc/

We investigate a reduction of supervised learning to game playing that reveals new connections and learning methods. For convex one-layer problems, we demonstrate an equivalence between global minimizers of the training problem and Nash equilibria in a simple game. We then show how the game can be extended to general acyclic neural networks with differentiable convex gates, establishing a bijection between the Nash equilibria and critical (or KKT) points of the deep learning problem. Based on these connections we investigate alternative learning methods, and find that regret matching can achieve competitive training performance while producing sparser models than current deep learning approaches.

Detailed comments 1. Line 36 introduces h(x) \phi(\theta x) where \phi is a transfer function, which I guess is intended to model the nonlinear activations in the output layer of the (single-layer) NN. Incorporating \phi breaks Theorems 1 and 2 since the loss \ell(\phi(\theta x), y) will not be a convex function of \theta for most choices of function \phi. I guess the authors meant to say "One linear layer learning games"? 2. Generality of results Regarding Theorems 4 and 5, the ancillary lemmas in the appendix assume that the activation functions f_v are convex and differentiable. This means that the most commonly used functions in practice (ReLU, maxout, max-pooling, sigmoid, and tanh units) are all ruled out along with many others. The results in the paper therefore apply in much less generality than a cursory reading would suggest.

We investigate a reduction of supervised learning to game playing that reveals new connections and learning methods. For convex one-layer problems, we demonstrate an equivalence between global minimizers of the training problem and Nash equilibria in a simple game. We then show how the game can be extended to general acyclic neural networks with differentiable convex gates, establishing a bijection between the Nash equilibria and critical (or KKT) points of the deep learning problem. Based on these connections we investigate alternative learning methods, and find that regret matching can achieve competitive training performance while producing sparser models than current deep learning approaches. Papers published at the Neural Information Processing Systems Conference.

We investigate a reduction of supervised learning to game playing that reveals new connections and learning methods. For convex one-layer problems, we demonstrate an equivalence between global minimizers of the training problem and Nash equilibria in a simple game. We then show how the game can be extended to general acyclic neural networks with differentiable convex gates, establishing a bijection between the Nash equilibria and critical (or KKT) points of the deep learning problem. Based on these connections we investigate alternative learning methods, and find that regret matching can achieve competitive training performance while producing sparser models than current deep learning approaches.