Adaptive Experimental Design for Policy Learning

This study designs an adaptive experiment for decision-making given multiple treatment arms, such as arms in slot machines, diverse therapies, and distinct unemployment assistance programs. The primary objective is to identify the best treatment arm for individuals given covariates, often referred to as a context, at the end of an experiment. Our problem is termed contextual fixed-budget best arm identification (BAI), an instance of the stochastic multi-armed bandit (MAB) problem (Thompson, 1933; Lai and Robbins, 1985). Our setting is a generalization of the fixed-budget BAI problem to minimize the expected simple regret at the end of a fixed number of rounds of an adaptive experiment, called a budget or sample size (Bubeck, Munos, and Stoltz, 2009, 2011; Audibert, Bubeck, and Munos, 2010). In our setting, at each round of an adaptive experiment, a decision-maker sequentially assigns one of the treatment arms to a research subject based on past observations and contextual information observed before the treatment assignment. At the end of the experiment, the experimenter recommends an estimated best treatment arm for future experimental subjects.