Rewards Structure in Games: Learning a Compact Representation for Action Space

Learning approximate payoff functions is important to understand the dynamics in multi-player interactions. In general repeat games, each player's payoff can be represented as a combination of all other players' action choices using normal forms, which grow exponentially as the number of action choices increases. Graphical games, however, provide a compact representation to specify the inter-relations where one player's action choice is influenced by its neighbourhood. In this paper, we present how to learn players' approximate payoff functions from normal-form representations, yet also learn a compact graphical game representation of the inter-relations among the players. In this normal form representation, we explore the structural connections of mutual influence between players' action choices in game playing. We formally describe the problem of learning a player influence network and give a novel reward structure-learning algorithm for multiagent graphical games, called the Multi-Descendent Regression Learning Structure Algorithm (MDRLSA). We evaluate MDRLSA on random graphical games generated in GAMUT. Experiments show that MDRLSA can efficiently identify the independence among players and extract the influence graph accurately. The running time of MDRLSA increases linearly with the number of strategy profiles of a game. Compared with state-of-the-art graphical game model learning methods, MDRLSA shows efficiency in terms of time and accuracy.