When Is Generalizable Reinforcement Learning Tractable?