A General Framework for Sample-Efficient Function Approximation in Reinforcement Learning