Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning

While ALE has enabled demonstration and evaluation of much more complex behaviors of deep RL agents, it Evaluation of deep reinforcement learning (RL) presents challenges as a suite of evaluation environments is inherently challenging. In particular, learned for topics on the frontier of deep RL. policies are largely opaque, and hypotheses about Challenge: Limited variation within games. Very little about the behavior of deep RL agents are difficult to individual games can be systematically altered, so ALE is test in black-box environments. Considerable effort poorly suited to testing how changes in the environment has gone into addressing opacity, but almost affect training and performance. New benchmarks such as no effort has been devoted to producing highquality OpenAI's Sonic the Hedgehog emulator and CoinRun inject environments for experimental evaluation environmental variation into the training schedule, while of agent behavior.