Functional Sequential Treatment Allocation with Covariates

The classical multi-armed bandit literature considers a sequential d ecision problem in which a policy maker attempts to assign subjects to the treatment with the highest expected outcome. Two practically relevant generalizations of this se tting have attracted much attention: (i) a problem where the decision maker can incorpor ate a vector of covariates in the assignment of each subject, cf. Woodroofe ( 1979), Yang et al. ( 2002), Rigollet and Zeevi ( 2010) and Perchet and Rigollet ( 2013); (ii) problems where instead of targeting the outcome distribution with highest expectation, the d ecision maker is interested in targeting another functional such as a quantile, a risk mea sure, or other characteristics of the distribution, cf.