Reviews: #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning

This paper is already available on arxiv and cited 10 times. It is a very good paper introducing a new approach to count-based exploration in deep reinforcement learning based on using binary hashcodes. The approach is interesting, the presentation is didactical, the results are good and the related literature is well covered. I learned a lot from reading this paper, so my only criticisms are on secondary aspects. I have few global points: - I think it would be interesting to first train the convolutional autoencoder, and only then use your exploration method to compare its performance to state-of-the-art methods.