#NeurIPS2022 outstanding paper – Gradient descent: the ultimate optimizer

Kartik Chandra, Audrey Xie, Jonathan Ragan-Kelley and Erik Meijer won a NeurIPS 2022 outstanding paper award for their work Gradient descent: the ultimate optimizer. Here, they tell us more about their work, the methodology and their main findings. Our paper studies the classic problem of "hyperparameter optimization". Nearly all of today's machine learning algorithms use a process called "stochastic gradient descent" (SGD) to train neural networks. SGD requires users to pick certain settings, or "hyperparameters," before running it.