For many years, research in AI plan generation was governed by a number of strong, simplifying assumptions: The planning agent is omniscient, its actions are deterministic and instantaneous, its goals are fixed and categorical, and its environment is static. More recently, researchers have developed expanded planning algorithms that are not predicated on such assumptions, but changing the way in which plans are formed is only part of what is required when the classical assumptions are abandoned. The demands of dynamic, uncertain environments mean that in addition to being able to form plans -- even probabilistic, uncertain plans -- agents must be able to effectively manage their plans. In this article, which is based on a talk given at the 1998 AAAI Fall Symposium on Distributed, Continual Planning, we first identify reasoning tasks that are involved in plan management, including commitment management, environment monitoring, alternative assessment, plan elaboration, metalevel control, and coordination with other agents. We next survey approaches we have developed to many of these tasks and discuss a plan-management system we are building to ground our theoretical work, by providing us with a platform for integrating our techniques and exploring their value in a realistic problem.

Disaster rescue is one of the most serious social issues that involves very large numbers of heterogeneous agents in the hostile environment. The intention of the RoboCup Rescue project is to promote research and development in this socially significant domain at various levels, involving multiagent teamwork coordination, physical agents for search and rescue, information infrastructures, personal digital assistants, a standard simulator and decision-support systems, evaluation benchmarks for rescue strategies, and robotic systems that are all integrated into a comprehensive system in the future. For this effort, which was built on the success of the RoboCup Soccer project, we will provide forums of technical discussions and competitive evaluations for researchers and practitioners. Although the rescue domain is intuitively appealing as a large-scale multiagent and intelligent system domain, analysis has not yet revealed its domain characteristics. The first research evaluation meeting will be held at RoboCup-2001, in conjunction with the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-2001), as part of the RoboCup Rescue Simulation League and RoboCup/AAAI Rescue Robot Competition.

In recent years, we have witnessed the success of autonomous agents applying machine-learning techniques across a wide range of applications. However, agents applying the same machine-learning techniques in online applications have not been so successful. Even agent-based hybrid recommender systems that combine information filtering techniques with collaborative filtering techniques have been applied with considerable success only to simple consumer goods such as movies, books, clothing, and food. Yet complex, adaptive autonomous agent systems that can handle complex goods such as real estate, vacation plans, insurance, mutual funds, and mortgages have emerged. To a large extent, the reinforcement learning methods developed to aid agents in learning have been more successfully deployed in offline applications. The inherent limitations in these methods have rendered them somewhat ineffective in online applications.

Process integrated mechanisms (PIM) offer a new approach to the problem of coordinating the activity of physically distributed systems or devices. Current approaches to coordination all have well-recognized strengths and weaknesses. We propose a novel architecture to add to the mix, called the Process Integrated Mechanism (PIM), which enjoys the advantages of having a single controlling authority while avoiding the structural difficulties that have traditionally led to its rejection in many complex settings. In many situations, PIMs improve on previous models with regard to coordination, security, ease of software development, robustness and communication overhead. In the PIM architecture, the components are conceived as parts of a single mechanism, even when they are physically separated and operate asynchronously.

Artificial intelligence (AI) research has explored a variety of problems and approaches since its inception, but for the last 20 years or so has been focused on the problems surrounding the construction of intelligent agents -- systems that perceive and act in some environment. In this context, "intelligence" is related to statistical and economic notions of rationality -- colloquially, the ability to make good decisions, plans, or inferences. The adoption of probabilistic and decision-theoretic representations and statistical learning methods has led to a large degree of integration and cross-fertilization among AI, machine learning, statistics, control theory, neuroscience, and other fields. The establishment of shared theoretical frameworks, combined with the availability of data and processing power, has yielded remarkable successes in various component tasks such as speech recognition, image classification, autonomous vehicles, machine translation, legged locomotion, and question-answering systems. As capabilities in these areas and others cross the threshold from laboratory research to economically valuable technologies, a virtuous cycle takes hold whereby even small improvements in performance are worth large sums of money, prompting greater investments in research.

This issue of AI Magazine brings together a collection of articles on challenges, mechanisms, and research progress in turn-taking and coordination between humans and machines. The contributing authors work in interrelated fields of spoken dialog systems, intelligent virtual agents, human-computer interaction, human-robot interaction, and semiautonomous collaborative systems and explore core concepts in coordinating speech and actions with virtual agents, robots, and other autonomous systems. Several of the contributors participated in the AAAI Spring Symposium on Turn-Taking and Coordination in Human-Machine Interaction, held in March 2015, and several articles in this issue are extensions of work presented at that symposium. The articles in the collection address key modeling, methodological, and computational challenges in achieving effective coordination with machines, propose solutions that overcome these challenges under sensory, cognitive, and resource restrictions, and illustrate how such solutions can facilitate coordination across diverse and challenging domains.

Back in the 1990s we started work on pedagogical agents, a new user interface paradigm for interactive learning environments. Pedagogical agents are autonomous characters that inhabit learning environments and can engage with learners in rich, face-to-face interactions. Building on this work, in 2000 we, together with our colleague, Jeff Rickel, published an article on pedagogical agents that surveyed this new paradigm and discussed its potential. We made the case that pedagogical agents that interact with learners in natural, life-like ways can help learning environments achieve improved learning outcomes. This article has been widely cited, and was a winner of the 2017 IFAAMAS Award for Influential Papers in Autonomous Agents and Multiagent Systems (IFAAMAS, 2017).

The ability to say "no" in a variety of ways and contexts is an essential part of being socio-cognitively human. Through a variety of examples, we show that, despite ominous portrayals in science fiction, AI agents with human-inspired noncompliance abilities have many potential benefits. Rebel agents are intelligent agents that can oppose goals or plans assigned to them, or the general attitudes or behavior of other agents. They can serve purposes such as ethics, safety, and task execution correctness, and provide or support diverse points of view. We present a framework to help categorize and design rebel agents, discuss their social and ethical implications, and assess their potential benefits and the risks they may pose.

Real-time traffic signal control presents a challenging multiagent planning pro blem, particularly in urban road networks where, unlike simpler arterial settings, there are competing dominant traffic flows that shift through the day. Further complicating matters, urban environments require attention to multimodal traffic flows (vehicles, pedestrians, bicyclists, buses) that move at different speeds and may be given different priorities. For the past several years, my research group has been developing and refining a real-time, adaptive traffic signal control system to address these challenges, referred to as scalable urban traffic control (Surtrac). Combining principles from automated planning and scheduling, multiagent systems, and traffic theory, Surtrac treats traffic signal control as a decentralized online planning process. In operation, each intersection repeatedly generates and executes (in rolling horizon fashion) signal-timing plans that optimize the movement of currently sensed approaching traffic through the intersection.