The Impact of Determinism on Learning Atari 2600 Games

Atari 2600 games are deterministic given a fixed policy leading to a fixed sequence of actions. This article investigates three methods for adding randomness: random initialization, epsilon-greedy action selection, and epislon-repeat action selection. These methods are evaluated by how well they are able to derail a memorizing agent without hurting the performance of a randomized agent. Results indicate that epsilon-repeat action selection best fits the desired criteria and lower values of epsilon than previously used are sufficient to derail the memorizing agent.