Navigating to the Best Policy in Markov Decision Processes