We study the computational tractability of PAC reinforcement learning with rich observations. We present new provably sample-efficient algorithms for environments with deterministic hidden state dynamics and stochastic rich observations. These methods operate in an oracle model of computation--accessing policy and value function classes exclusively through standard optimization primitives--and therefore represent computationally efficient alternatives to prior algorithms that require enumeration.

We propose a new oracle-based algorithm, BISTRO+, for the adversarial contextual bandit problem, where either contexts are drawn i.i.d. or the sequence of contexts is known a priori, but where the losses are picked adversarially.

We study the computational tractability of PAC reinforcement learning with rich observations. We present new provably sample-efficient algorithms for environments with deterministic hidden state dynamics and stochastic rich observations. These methods operate in an oracle model of computation--accessing policy and value function classes exclusively through standard optimization primitives--and therefore represent computationally efficient alternatives to prior algorithms that require enumeration.