Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks

High-dimensional problems are prominent across all corners of science and industrial applications. Within this realm, optimizing black-box functions and operators can be computationally expensive and require large amounts of hardto-obtain data for training surrogate models. Uncertainty quantification becomes a key element in this setting, as the ability to quantify what a surrogate model does not know offers a guiding principle for new data acquisition. However, existing methods for surrogate modeling with built-in uncertainty quantification, such as Gaussian Processes (GPs) [1], have demonstrated difficulty in modeling problems that exist in high dimensions. While other methods such as Bayesian neural networks [2] (BNNs) and deep ensembles [3] are able to mitigate this issue, their computational cost can still be prohibitive for some applications. This problem becomes more prominent in Operator Learning, where either inputs or outputs of a model are functions residing in infinite-dimensional function spaces. The field of Operator Learning has had many advances in recent years[4, 5, 6, 7, 8, 9], with applications across many domains in the natural sciences and engineering, but so far its integration with uncertainty quantification is limited [10, 11]. In addition to safety-critical problems using deep learning such as ones in medicine [12, 13] and autonomous driving [14], the generation of uncertainty measures can also be important for decision making when collecting new data in the physical sciences. Total uncertainty is often made up of two distinct parts: epistemic and aleatoric uncertainty.