Explicit Planning for Efficient Exploration in Reinforcement Learning