Global optimization of expensive, potentially gradient-free functions has long been a critical component of many complex problems in science and engineering.

We investigate the problem of reliably assessing group fairness when labeled examples are few but unlabeled examples are plentiful.

Suppose an online platform wants to compare a treatment and control policy, e.g.,

Bayesian model evidence gives a clear criteria for such model selection. However, computing model evidence requires integration over the likelihood, which is challenging, particularly when the likelihood is non-closed-form and/or expensive.

Let defined denote conditional (x)= P(y =1|x).