PyGlove: SymbolicProgramming forAutomatedMachineLearning

Neural networks are sensitive to architecture and hyper-parameter choices [3,4]. For example, on the ImageNet dataset [5], we have observed a large increase in accuracy thanks to changes in architectures, hyper-parameters, and training algorithms, from the seminal work of AlexNet [5] to recent state-of-the-art models such as EfficientNet [6]. However, as neural networks become increasingly complex,thepotential number ofarchitecture and hyper-parameter choices becomes numerous.