Quantile (and, more generally, KL) regret bounds, such as those achieved by NormalHedge (Chaudhuri, Freund, and Hsu 2009) and its variants, relax the goal of competing against the best individual expert to only competing against a majority of experts on adversarial data.

However, they suffer from an exponential complexity in terms of the input dimension of the problem. In order to address this issue, we propose a novel variant of random search that exploits negative curvature by only relying on function evaluations.

D.5 Metamath Lean versions To compare our models in the same setup while working on this project, we ran all our experiments

We sincerely thank all reviewers for their valuable comments and we address individual questions below. R1: This paper only compares to the 2 years ago's work (DARTS). R2: ENAS/DARTS are fairly different architecture search algorithms. R2: It would be nice to have an explicit related work section. We will move the related work to main content.