Our design policy successfully achieves amortized intervention design on the distribution of the training environment while also generalizing well to distribution shifts in test-time design environments.

Suppose an online platform wants to compare a treatment and control policy, e.g.,

For example, in personalized medical diagnostics, a model is built based on historical data to facilitate an optimal treatment for a new patient (Bica et al., 2021).

A/B testing is critical for modern technological companies to evaluate the effectiveness of newly developed products against standard baselines. This paper studies optimal designs that aim to maximize the amount of information obtained from online experiments to estimate treatment effects accurately.

In this work, we address the more challenging yet realistic setting of few-shot experimental design, where only a few labeled data points of input designs and their corresponding values are available.

BACON allows computational resources to be focused on simulations that provide the most value in terms of reducing epistemic uncertainty.

Typically, treatment is assigned in a nonadaptive manner, where assignments are determined before any outcomes are observed.