Neural Information Processing Systems http://nips.cc/

We study the generalization performance of full-batch optimization algorithms for stochastic convex optimization: these are first-order methods that only access the exact gradient of the empirical risk (rather than gradients with respect to individual data points), that include a wide range of algorithms such as gradient descent, mirror descent, and their regularized and/or accelerated variants.

In addition, we examine two common computation frameworks for this problem, i.e., population-based training (PBT) and iterative learning

A classical problem that fits in the framework of (1) is the best subset selection problem in linear regression [6,20].

Typical examples are hyperparameters selection [5, 38, 17, 6], data augmentation [11, 42], implicit deep learning [3] or neural architecture search [33]. Figure 1: Convergence curves of the two proposed methods on a toy problem.