Expert Selection in High-Dimensional Markov Decision Processes

Abstract-- In this work we present a multi-armed bandit framework for online expert selection in Markov decision processes and demonstrate its use in high-dimensional settings. Our method takes a set of candidate expert policies and switches between them to rapidly identify the best performing expert using a variant of the classical upper confidence bound algorithm, thus ensuring low regret in the overall performance of the system. This is useful in applications where several expert policies may be available, and one needs to be selected at runtime for the underlying environment. Markov decision processes (MDPs) represent a mathematical reach a set of high-reward states. For the high-dimensional framework for dealing with decision problems in case, we will use the Seaquest video game environment, many fields. It is usually hard, however, to predict how shown in Figure 1, where each expert policy is trained under changes in the underlying MDP might affect the performance different observation models.