Counterfactual regret minimization is a family of algorithms of no-regret learning dynamics capable of solving large-scale imperfect information games. We propose implementing this algorithm as a series of dense and sparse matrix and vector operations, thereby making it highly parallelizable for a graphical processing unit, at a cost of higher memory usages. Our experiments show that our implementation performs up to about 352.5 times faster than OpenSpiel's Python implementation and up to about 22.2 times faster than OpenSpiel's C++ implementation and the speedup becomes more pronounced as the size of the game being solved grows. Counterfactual regret minimization (CFR) (Zinkevich et al., 2007) is a family of algorithms of noregret learning dynamics capable of solving large-scale imperfect information games. Its variants dominated the development of AI agents for large imperfect information games like Poker (Tammelin et al., 2015; Moravčík et al., 2017; Brown & Sandholm, 2018; 2019b) and The Resistance: Avalon (Serrino et al., 2019) and were components of ReBeL (Brown et al., 2020) and student of games (Schmid et al., 2023).

LiteEFG is an efficient library with easy-to-use Python bindings, which can solve multiplayer extensive-form games (EFGs). LiteEFG enables the user to express computation graphs in Python to define updates on the game tree structure. The graph is then executed by the C++ backend, leading to significant speedups compared to running the algorithm in Python. Moreover, in LiteEFG, the user needs to only specify the computation graph of the update rule in a decision node of the game, and LiteEFG will automatically distribute the update rule to each decision node and handle the structure of the imperfect-information game.

The OpenSpiel framework provides a collection of environments and algorithm implementations for studying Reinforcement Learning (RL) in games. OpenSpiel includes many popular general-sum, zero-sum, perfect and imperfect information games with episodic interfaces suitable for training RL agents. The algorithms implemented in OpenSpiel are contemporary or state-of-the-art (SOTA) and are designed to be highly configurable and extensible. As stated in the documentation and provided example code, the given default parameters are (in the majority of cases) intended to solve the imperfect information poker variant Kuhn [2]. However, the papers originally proposing many of the OpenSpiel algorithms may not necissarily provide results for this environment and instead report results for more challenging games such as Leduc or Heads up No-Limit Texas Holdem. This limits OpenSpiel users' ability to convinently verify the correctness and performance of algorithim implementations using this tool.

Deep reinforcement learning(DRL) has been at the center of some of the biggest breakthroughs of artificial intelligence(AI) in the last few years. However, despite all its progress, DRL methods remain incredibly difficult to apply in mainstream solutions given the lack of tooling and libraries. Consequently, DRL remains mostly a research activity that hasn't seen a lot of adoption into real world machine learning solutions. Addressing that problem requires better tools and frameworks. Among the current generation of artificial intelligence(AI) leaders, DeepMind stands alone as the company that has done the most to advance DRL research and development. Recently, the Alphabet subsidiary has been releasing a series of new open source technologies that can help to streamline the adoption of DRL methods.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games. OpenSpiel supports n-player (single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and sequential, strictly turn-taking and simultaneous-move, perfect and imperfect information games, as well as traditional multiagent environments such as (partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel also includes tools to analyze learning dynamics and other common evaluation metrics. This document serves both as an overview of the code base and an introduction to the terminology, core concepts, and algorithms across the fields of reinforcement learning, computational game theory, and search.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games. OpenSpiel supports n-player (single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and sequential, strictly turn-taking and simultaneous-move, perfect and imperfect information games, as well as traditional multiagent environments such as (partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel also includes tools to analyze learning dynamics and other common evaluation metrics. Games are represented as procedural extensive-form games, with some natural extensions. The core API and games are implemented in C and exposed to Python.

Reinforcement learning, the AI training technique that's brought to fruition systems capable of defeating world poker champions and guiding self-driving cars, isn't the simplest thing in the world to wrangle. That's particularly true in the gaming domain, where cutting-edge approaches sometimes require bespoke tools that aren't publicly available. In a paper recently published on the preprint server Arxiv.org, At its core, it's a collection of environments and algorithms for research in general reinforcement learning and search and planning in games, with tools to analyze learning dynamics and other common evaluation metrics. "The purpose of OpenSpiel is to promote general multiagent reinforcement learning across many different game types, in a similar way as general game-playing but with a heavy emphasis on learning and not in competition form," wrote the researchers.