Learning Mean-Field Games through Mean-Field Actor-Critic Flow

Mean-field games (MFGs), introduced independently by Lasry and Lions [39, 40, 41] and by Huang, Caines, and Malham e [32, 31], provide a powerful framework for modeling strategic interactions among a large population of agents, where each agent responds to the aggregate distribution of the population rather than to individual players. Over the past decade, substantial progress has been made in the theoretical development of MFGs, including the well-posedness of equilibria under monotonicity conditions [39], and the rigorous connection to McKean-Vlasov forward-backward stochastic differential equations (FBSDEs) [16] and master equations [14]. A broader exposition of the theory and its historical development can be found in [13, 10, 25, 17]. From a computational perspective, solving MFGs remains challenging due to their intrinsic infinite-dimensional structure arising from the dependence on the evolving population distribution. Classical numerical approaches focus on solving the coupled Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck (FP) equations directly [1]. More recent advances leverage deep learning techniques to approximate the partial differential equation (PDE) systems [49, 9], FBSDEs [19, 24, 28], and even master equations [21, 26]. In parallel, reinforcement learning (RL)-based approaches have attracted growing attention for solving MFGs, motivated by their model-free nature, i.e., the ability to learn optimal strategies directly from observations without requiring explicit knowledge of the system dynamics [27, 48, 5, 4]. We refer interested readers to the recent survey [42]. In this work, we propose the Mean-Field Actor-Critic (MFAC) flow, a learning-based framework for solving MFGs with general distribution dependence.