In recent years, research on Automated Machine Learning (AutoML) [1] has made great strides in the data-driven design of neural network architectures [2, 3] and training hyperparameters [4].

Neural Information Processing Systems http://nips.cc/

The model combines aprobabilistic encoder withamulti-task model suchthatitcan generate inexpensive and realistic tasks of the class of problems of interest.

Weempirically assess theimpact oftheseregularization cocktailsforMLPs ina large-scale empirical study comprising 40 tabular datasets and demonstrate that (i) well-regularized plain MLPs significantly outperform recent state-of-the-art specialized neural network architectures, and (ii) they even outperform strong traditionalMLmethods,suchasXGBoost.

Computational modeling and machine learning (ML) have led to an acceleration of scientific innovation in diverse areas, ranging from drug design to robotics to material science.

Toaddress theabovechallenges, wepropose anovelhyperparameter mutation (HPM) scheduling algorithm in this study, which adopts a population based training framework to explicitly learn a trade-off (i.e., a mutation schedule) between using the hypergradient-guided local search and the mutation-driven global search.

Neural Architecture Search (NAS) wasfirst proposed toachievestate-of-the-art performance through thediscoveryofnewarchitecture patterns, without human intervention.

Bayesian optimization (BO) is a popular approach to optimize expensive-toevaluate black-box functions. A significant challenge in BO is to scale to highdimensional parameter spaces whileretaining sample efficiency. Asolution considered in existing literature is to embed the high-dimensional space in a lowerdimensional manifold, often via a random linear embedding.