Reinforcement learning utilizing kernel ridge regression to predict the expected value function represents a powerful method with great representational capacity. This setting is a highly versatile framework amenable to analytical results. Weconsider kernel-based function approximation for RL in the infinite horizon average reward setting, also referred toasthe undiscounted setting. Wepropose an optimistic algorithm, similar to acquisition function based algorithms in the special caseofbandits.

Theexpected performance objective in (13) is equivalent setting toβ = 0 in Figure 1a. The algorithm gets stuck and repeatedly evaluates inputs (orange crosses) at alocal optimum of the true objectivefunction (black dashed).