Coevolution of cognition and cooperation in structured populations under reinforcement learning

The evolution of cooperation has been investigated intensely in various disciplines, such as biology, economics, computer science, physics and psychology. There are two important dimensions, among many (Bowles and Gintis, 2011; Lehmann and Keller, 2006; Nowak, 2006), that have been shown to affect the evolution of cooperation: the interaction structure, i.e., who interacts with whom (Santos et al., 2006), and the mode of cognition, i.e., the extent of deliberation as opposed to intuition (Capraro, 2019). While for the interaction structure there is a substantial consensus that sparse and heavily clustered networks help the spread of cooperation (Nowak, 2006; Ohtsuki et al., 2006), for the mode of cognition results are more articulated and depend on specific features of the social dilemma (Bear et al., 2017; Bear and Rand, 2016) and of the cost of deliberation (Jagau and van Veelen, 2017). An important aspect in evolutionary models is the behavioral rule adopted by agents, which heavily contributes to determining the trajectories of the dynamic adjustment. While the literature has extensively considered behavioral rules encompassing best reply (Bilancini and Boncinelli, 2009) and imitation (Levine and Pesendorfer, 2007) as well as processes of the type death-birth or birth-death (Ohtsuki et al., 2006), little attention has been given to evolutionary dynamics based on reinforcement learning (Tanabe and Masuda, 2012). Reinforcement learning is a prominent behavioral rule originated in behavioral sciences (Skinner, 1938a,b) and recently become extremely popular in computer sciences, with many different applications (Nian et al., 2020).