A key to the success of MAS is efficient and effective multiagent learning (MAL). The past 25 years have seen a great interest and tremendous progress in the field of MAL. This article introduces and overviews this field by presenting its fundamentals, sketching its historical development, and describing some key algorithms for MAL. Moreover, main challenges that the field is facing today are identified. These agents may be computer programs, robots, or even humans.

The research addressed in the autonomous agents field covers a wide spectrum of levels from the cognitive to the organizational, exploits diverse mechanisms and approaches, and has had a major impact on many aspects of artificial intelligence research. In 2011 the Autonomous Agents and Multiagent Systems (AAMAS) conference series celebrated its 10th anniversary, having begun as the successful merger of three related events that had run for some years previously. The 2011 AAMAS conference received 575 submissions, and 126 papers were selected for publication as full papers. Representation under all submissions of topics (measured by first keyword) was broad, with top counts in areas such as teamwork, coalition formation, and coordination (31), distributed problem solving (30), game theory (30), planning (26), multiagent learning (24), and trust, reliability, and reputation (17). The tag cloud (figure 1), generated from the titles of the full papers at the conference, conveys a sense of the relative prominence of topics.

It was started in 2005 and is an annual event that attracts between 5 and 10 teams. It has since been organized by the AI group at Clausthal University of Technology. MAPC is not collocated with any other event. Using our MASSim platform, the participants are running their own systems locally and only interact with the tournament server over the Internet. A steering committee oversees the whole process and determines the organization committee. The scenario changes every other year: the current one is "Agents on Mars." The goal was to implement a team of heterogeneous, cooperating agents to occupy zones on planet Mars. The infrastructure on Mars is given by a directed graph (300 nodes). Agents could take on roles (explorer, sentinel, saboteur, repairer, inspector) and needed to cooperate in an environment with incomplete knowledge so as to win against a competing team: the graph was not known, and each action comes at a price. Conquered terrain brings in money to improve agents. The timeline of the contest is as follows.

Toward that end, this article presents PROTECT, a game-theoretic system deployed by the United States Coast Guard (USCG) in the Port of Boston for scheduling its patrols. USCG has termed the deployment of PROTECT in Boston a success; PROTECT is currently being tested in the Port of New York, with the potential for nationwide deployment. PROTECT is premised on an attackerdefender Stackelberg game model and offers five key innovations. First, this system is a departure from the assumption of perfect adversary rationality noted in previous work, relying instead on a quantal response (QR) model of the adversary's behavior -- to the best of our knowledge, this is the first real-world deployment of the QR model. Second, to improve PROTECT's efficiency, we generate a compact representation of the defender's strategy space, exploiting equivalence and dominance.

Instead, patrol units move about the transit system, inspecting the tickets of passengers, who face fines if caught fare evading. The deterrence of fare evasion depends on the unpredictability and effectiveness of the patrols. TRUSTS models the problem of computing patrol strategies as a leader-follower Stackelberg game where the objective is to deter fare evasion and hence maximize revenue. This problem differs from previously studied Stackelberg settings in that the leader strategies must satisfy massive temporal and spatial constraints; moreover, unlike in these counterterrorism-motivated Stackelberg applications, a large fraction of the ridership might realistically consider fare evasion, and so the number of followers is potentially huge. A third key novelty in our work is deliberate simplification of leader strategies to make patrols easier to execute. We present an efficient algorithm for computing such patrol strategies and present experimental results using real-world ridership data from the Los Angeles Metro Rail system. The Los Angeles County Sheriff's Department is currently carrying out trials of TRUSTS. There are, quite literally, no barriers to entry, as illustrated in figure 1. Instead, security personnel are dynamically deployed throughout the transit system, randomly inspecting passenger tickets. This proof-of-payment fare collection method is typically chosen as a more cost-effective alternative to direct fare collection, that is, when the revenue lost to fare evasion is believed to be less than what it would cost to make fare evasion impossible. For the LA Metro, with approximately 300,000 riders daily, this revenue loss can be significant; the annual cost has been estimated at $5.6 million. The Los Angeles Sheriff's Department (LASD) deploys uniformed patrols onboard trains and at stations for fare checking (and for other purposes such as crime prevention), in order to discourage fare evasion.

A more modern view is to envision drivers and passengers as actively interacting with a complex automated system. Such interactive activity leads us to consider intelligent and advanced ways of interaction leading to cars that can adapt to their drivers. In this article, we focus on the adaptive cruise control (ACC) technology that allows a vehicle to automatically adjust its speed to maintain a preset distance from the vehicle in front of it based on the driver's preferences. Although individual drivers have different driving styles and preferences, current systems do not distinguish among users. We introduce a method to combine machine-learning algorithms with demographic information and expert advice into existing automated assistive systems.

The experience gained during these competitions has increased the maturity level of the field, which allowed deploying robots after real disasters (for example, the Fukushima Daiichi nuclear disaster). This article provides an overview of these competitions and highlights the state of the art and the lessons learned. A major outcome of this initiative was the RoboCup Rescue competitions. In this article, we introduce the RoboCup Rescue leagues, namely the Rescue Robot League (RRL) and the Rescue Simulation League (RSL) (Tadokoro et al. 2000; Kitano and Tadokoro 2001). Disaster mitigation is an important social issue involving large numbers of heterogeneous agents acting in hostile environments.

Intelligent game-based learning environments integrate commercial game technologies with AI methods from intelligent tutoring systems and intelligent narrative technologies. This article introduces the Crystal Island intelligent game-based learning environment, which has been under development in the authors' laboratory for the past seven years. After presenting Crystal Island, the principal technical problems of intelligent game-based learning environments are discussed: narrative-centered tutorial planning, student affect recognition, student knowledge modeling, and student goal recognition. The burgeoning field of game-based learning has made significant advances, including theoretical developments (Gee 2007), as well as the creation of gamebased learning environments for a broad range of K-12 subjects (Habgood and Ainsworth 2011; Ketelhut et al. 2010; Warren, Dondlinger, and Barab 2008) and training objectives (Johnson 2010; Kim et al. 2009). Of particular note are the results of recent empirical studies demonstrating that in addition to game-based learning environments' potential for motivation, they can enable students to achieve learning gains in controlled laboratory settings (Habgood and Ainsworth 2011) as well as classroom settings (Ketelhut et al. 2010).

In this article, we focus on RoboCup robot soccer, and present its five current leagues, which address complementary scientific challenges through different robot and physical setups. Full details on the status of the RoboCup soccer leagues, including league history and past results, upcoming competitions, and detailed rules and specifications are available from the league homepages and wikis. At that time -- the mid 1990s -- there were very few effective mobile robots, and the Honda P2 humanoid robot had just been presented to the public for the first time. The RoboCup challenge, set as an adversarial game between teams of autonomous robots, was therefore a fascinating and exciting development as well as a reference problem for AI. RoboCup introduced three robot soccer leagues, Simulation, Small Size, and Middle Size, and organized its first competitions at the 1997 International Joint Conference on Artificial Intelligence, held in Nagoya, Japan.

Each political decision in fact implies some form of social reactions, it affects economic and financial aspects and has substantial environmental impacts. Improving decision making in this context could have a huge beneficial impact on all these aspects. There are a number of Artificial Intelligence techniques that could play an important role in improving the policy-making process such as decision support and optimization techniques, game theory, data and opinion mining and agent-based simulation. We outline here some potential use of AI technology as it emerged by the European Union (EU) EU FP7 project ePolicy: Engineering the Policy Making Life Cycle, and we identify some potential research challenges. They are extremely complex, occur in rapidly changing environments characterized by uncertainty, and involve conflicts among different interests.