High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling

We introduce a novel semi-supervised learning approach, named Teacher-Student Bayesian Optimization (TSBO), integrating the teacher-student paradigm into BO to minimize expensive labeled data queries for the first time. TSBO incorporates a teacher model, an unlabeled data sampler, and a student model. The student is trained on unlabeled data locations generated by the sampler, with pseudo labels predicted by the teacher. The interplay between these three components implements a unique selective regularization to the teacher in the form of student feedback. This scheme enables the teacher to predict high-quality pseudo labels, enhancing the generalization of the Figure 1: Visualization of queried data (dots) and trends GP surrogate model in the search space. To fully (arrow sequences) on a high-dimensional molecule design exploit TSBO, we propose two optimized unlabeled task (Sterling & Irwin, 2015) to maximize the Penalized data samplers to construct effective student LogP score (Gómez-Bombarelli et al., 2018). Red and blue feedback that well aligns with the objective of colors represent TSBO and a baseline (with vanilla BO), respectively.