Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces

However, real-world optimisation problems High-dimensional black-box optimisation remains are often neither low-dimensional nor continuous: many an important yet notoriously challenging large-scale practical problems exhibit complex interactions problem. Despite the success of Bayesian among high-dimensional input variables, and are optimisation methods on continuous domains, often categorical in nature or involve a mixture of both domains that are categorical, or that mix continuous and categorical input variables. An example continuous and categorical variables, remain of the former is the maximum satisfiability problem, challenging. We propose a novel solution whose exact solution is np-hard (Creignou et al., 2001), - we combine local optimisation with a tailored and an example for the latter is the hyperparameter kernel design, effectively handling highdimensional tuning for a deep neural network: the optimisation categorical and mixed search scope comprise both continuous hyperparameters, e.g., spaces, whilst retaining sample efficiency. We learning rate and momentum, and categorical ones, further derive convergence guarantee for the e.g., optimiser type {sgd, Adam,...} and learning rate proposed approach. Finally, we demonstrate scheduler type {step decay, cosine annealing}.