The goal of our work was easy reproducibility and clearly showing the benefits of learning to explore over the state