This paper considers the problem of designing optimal algorithms for reinforcement learning in two-player zero-sum games. We focus on self-play algorithms which learn theoptimal policy by playing againstitself without any direct supervision.

Our experiments show that Genfer is often faster than the existing exact inference tools PSI, Dice, and Prodigy.

A finite episodic Markov decision process (MDP) is a tuple (S,A,H,α,P,r) where S and A are the finite sets of states and actions withS = |S|,A = |A|, H is the (fixed) episode length andα is the initial state distribution.