Temporally-extended actions have been widely used in both learning and planning to improve scalability and reduce complexity.

Focusing learning on these relevant actions can significantly improve training efficiency and effectiveness.

A significant challenge in extending this success to mass production is the variation between instances of the problem, e.g.,

A significant challenge in extending this success to mass production is the variation between instances of the problem, e.g.,

However, the nonlinearity of risk measures makes it challenging to achieve convergence and optimality.

We provide a closed-form expression for the worst occupation measure.

Datasets often suffer severe selection bias; clinical labels are only available on patients for whom doctors ordered medical exams. To assess model performance outside the support of available data, we present a computational framework for adaptive labeling, providing cost-efficient model evaluations under severe distribution shifts. We formulate the problem as a Markov Decision Process over states defined by posterior beliefs on model performance. Each batch of new labels incurs a "state transition" to sharper beliefs, and we choose batches to minimize uncertainty on model performance at the end of the label collection process. Instead of relying on high-variance REINFORCE policy gradient estimators that do not scale, our adaptive labeling policy is optimized using path-wise policy gradients computed by auto-differentiating through simulated roll-outs. Our framework is agnostic to different uncertainty quantification approaches and highlights the virtue of planning in adaptive labeling. On synthetic and real datasets, we empirically demonstrate even a one-step lookahead policy substantially outperforms active learning-inspired heuristics.

A major challenge in the analysis of multi-agent systems is the restriction on joint policies of agents.