Inmultiagent domains, coping withnon-stationary agents thatchange behaviors from time to time is a challenging problem, where an agent is usually required to be able to quickly detect the other agent's policy during online interaction, and then adapt its own policy accordingly.

Value function factorization (VFF) is a popular approach to cooperative multiagent reinforcement learning in order to learn local value functions from global rewards.

We appreciate R1's recognition of the novelty of our contribution to MARL and the potential impact on a We address R1's two concerns below. "give-reward" actions are direct applications of conventional RL (which have been applied to multi-agent incentivization We appreciate R2's positive feedback on our quantitative results and we are glad that our behavioral Figure 6b where the agent gives nonzero reward for "fire cleaning beam but miss" after 40k steps, one reason is that the Figure 6a), so it may have "forgotten" the difference between successful and unsuccessful usage of the cleaning beam. As demonstrated more clearly in the Escape Room results (e.g. We thank R3 for recognizing our contribution to the general class of opponent-shaping algorithms. Prisoner's Dilemma is fully observable).

Game-theoretic solution concepts, such as the Nash equilibrium, have been key to finding stable joint actions in multi-player games. However, it has been shown that the dynamics of agents' interactions, even in simple two-player games with few strategies, are incapable of reaching Nash equilibria, exhibiting complex and unpredictable behavior. Instead, evolutionary approaches can describe the long-term persistence of strategies and filter out transient ones, accounting for the long-term dynamics of agents' interactions. Our goal is to identify agents' joint strategies that result in stable behavior, being resistant to changes, while also accounting for agents' payoffs, in dynamic games. Towards this goal, and building on previous results, this paper proposes transforming dynamic games into their empirical forms by considering agents' strategies instead of agents' actions, and applying the evolutionary methodology $\alpha$-Rank to evaluate and rank strategy profiles according to their long-term dynamics. This methodology not only allows us to identify joint strategies that are strong through agents' long-term interactions, but also provides a descriptive, transparent framework regarding the high ranking of these strategies. Experiments report on agents that aim to collaboratively solve a stochastic version of the graph coloring problem. We consider different styles of play as strategies to define the empirical game, and train policies realizing these strategies, using the DQN algorithm. Then we run simulations to generate the payoff matrix required by $\alpha$-Rank to rank joint strategies.

Achieving and maintaining cooperation between agents to accomplish a common objective is one of the central goals of Multi-Agent Reinforcement Learning (MARL). Nevertheless in many real-world scenarios, separately trained and specialized agents are deployed into a shared environment, or the environment requires multiple objectives to be achieved by different coexisting parties. These variations among specialties and objectives are likely to cause mixed motives that eventually result in a social dilemma where all the parties are at a loss. In order to resolve this issue, we propose the Incentive Q-Flow (IQ-Flow) algorithm, which modifies the system's reward setup with an incentive regulator agent such that the cooperative policy also corresponds to the self-interested policy for the agents. Unlike the existing methods that learn to incentivize self-interested agents, IQ-Flow does not make any assumptions about agents' policies or learning algorithms, which enables the generalization of the developed framework to a wider array of applications. IQ-Flow performs an offline evaluation of the optimality of the learned policies using the data provided by other agents to determine cooperative and self-interested policies. Next, IQ-Flow uses meta-gradient learning to estimate how policy evaluation changes according to given incentives and modifies the incentive such that the greedy policy for cooperative objective and self-interested objective yield the same actions. We present the operational characteristics of IQ-Flow in Iterated Matrix Games. We demonstrate that IQ-Flow outperforms the state-of-the-art incentive design algorithm in Escape Room and 2-Player Cleanup environments. We further demonstrate that the pretrained IQ-Flow mechanism significantly outperforms the performance of the shared reward setup in the 2-Player Cleanup environment.

A virtual edition of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS) conference was held on 9-13 May. Videos of the talks are now available for public viewing, and you can also see the sessions from the various workshops. Alison is interested in how cities work and builds spatial agent-based models (ABMs) to study how people move around and how behaviour plays out in space and time. There are a number of challenges with these kinds of models and they need to be really robust if they are to be adopted by policy makers. So, why should we be interested in modelling cities?

The 2005 Autonomous Agents and Multiagent Systems Conference (AAMAS 2005) was held July 25-29, 2005, at the University of Utrecht, the Netherlands. This report reviews the activities of that conference, including the workshop and tutorial programs, the main conference and poster tracks, the industry paper track, the demonstration track and sponsor demonstration sessions, the invited talks, exhibition, doctoral mentoring program, as well the sponsorship and scholarships activities. The Autonomous Agents and Multiagent Systems (AAMAS) conference series is the main conference venue for research in this area. It was initiated in 2002 as a merger of three conferences: the International Conference on Autonomous Agents, the International Conference on Multiagent Systems, and the International Workshop on Agent Theories, Architectures, and Languages. It aims to provide a highprofile and high-quality forum for research in the theory and practice of autonomous agents and multiagent systems.

I Have a Robot, and I'm Not Afraid to Use It! In this article, I submit that the growing success of robotics at AAMAS is due not only to the nurturing efforts of the AAMAS community, but mainly to the increasing recognition of an important, deeper, truth: it is scientifically useful to roboticists and agent researchers to think of robots as agents. Today, there is a resurgent interest and recognition of the importance of robotics research framed within areas of research familiar to autonomous agents and multiagent systems researchers. Robots (and roboticists) increasingly appear at the AAMAS conferences, and for a good reason. The AAMAS community is investing efforts to encourage robotics research within itself.

Agent-Oriented Software Engineering (AOSE) is an emerging software engineering paradigm that advocates the application of best practices in the development of Multi-Agent Systems (MAS) through the use of agents and organizations of agents. This paper outlines the MADE system, which provides an interactive platform for people who are not well-versed in AOSE to contribute to the rapid prototyping of MASs with ease.