Review for NeurIPS paper: The NetHack Learning Environment

Strengths: The main strength of the paper is in the environment, which will certainly be useful for the RL/embodied AI community. The NetHack environment proposed in the paper seems to fill a gap in exiting environments for RL research, which can help develop new RL algorithms, but also new problems related to embodied intelligence. The environment is procedurally generated and stochastic, which avoids having agents memorizing past episodes in order to solve the game, and makes some of the existing exploration methods such as Go-Explore fail. While the observations are symbolic, they contain a large number of symbols corresponding to the different game elements, as well as natural language, creating opportunities for combining NLP and RL. The game entities are compositional, meaning that agents can reason about common attributes to interact with entities of different classes (line 108).