Multi-agent Reinforcement Learning in OpenSpiel: A Reproduction Report

The OpenSpiel framework provides a collection of environments and algorithm implementations for studying Reinforcement Learning (RL) in games. OpenSpiel includes many popular general-sum, zero-sum, perfect and imperfect information games with episodic interfaces suitable for training RL agents. The algorithms implemented in OpenSpiel are contemporary or state-of-the-art (SOTA) and are designed to be highly configurable and extensible. As stated in the documentation and provided example code, the given default parameters are (in the majority of cases) intended to solve the imperfect information poker variant Kuhn [2]. However, the papers originally proposing many of the OpenSpiel algorithms may not necissarily provide results for this environment and instead report results for more challenging games such as Leduc or Heads up No-Limit Texas Holdem. This limits OpenSpiel users' ability to convinently verify the correctness and performance of algorithim implementations using this tool.