This paper proposes a fast and scalable method for uncertainty quantification of machine learning models' predictions.

One approach toward automatic grading is to learn an agent that interacts with a student's program and explores states indicative of errors via reinforcement learning. However, existing work on this approach only provides binary feedback of whether a program is correct or not, while students require finer-grained feedback on the specific errors in their programs to understand their mistakes. In this work, we show that exploring to discover errors can be cast as a meta-exploration problem.

Our goal is to learn an optimal policy that is also permutation invariant.

Therefore, a variant known as sparse PCA is often preferred.

We present the problem of reinforcement learning with exogenous termination. We define the Termination Markov Decision Process (TerMDP), an extension of the MDP framework, in which episodes may be interrupted by an external non-Markovian observer.

These RL algorithms are designed with a flat Markovian view of the environment in the form of a "state, action, reward, and next state" interface [

Observations (DMSO) framework introduced in Foster et al.

We propose a general framework to design posterior sampling methods for model-based RL. We show that the proposed algorithms can be analyzed by reducing regret to Hellinger distance in conditional probability estimation. We further show that optimistic posterior sampling can control this Hellinger distance, when we measure model error via data likelihood. This technique allows us to design and analyze unified posterior sampling algorithms with state-of-the-art sample complexity guarantees for many model-based RL settings. We illustrate our general result in many special cases, demonstrating the versatility of our framework.

QB construction for the one-dimensional marginals and model the dependence using highly expressive vine copulas.