Heuristic search is a core area of Artificial Intelligence with applications to planning, scheduling and game playing. Real-time heuristic search applies to search problems where plan execution needs to start before a complete solution can be computed. Since the inception of real-time heuristic search in the early 1990s a great number of algorithms have been proposed and evaluated. In this paper we break them down into building blocks and conduct a search in the space of such building blocks. Even simple tabulated and iterative searches find new real-time heuristic search algorithms outperforming manually crafted contemporary algorithms.

The agent operates in a real-time setting by interleaving local planning, learning and move execution. In this paper we propose a simple parametric algorithm that combines weighting with learning from multiple neighbors. Doing so breaks heuristic admissibility but allows the agent to escape heuristic depressions more quickly. We prove completeness of the algorithm and empirically compare it to several competitors more than twenty years apart. In a large-scale evaluation the new algorithm found better solutions than the recent algorithms, despite not learning additional information that they do. Finally, we study robustness of the algorithms to noise in the heuristic function -- a desirable property in a physical implementation of real-time heuristic search. The new algorithm outperforms its contemporaries.

Real-time heuristic search algorithms obey a constant limit on planning time per move. Agents using these algorithms can execute each move as it is computed, suggesting a strong potential for application to real-time video-game AI. Recently, a breakthrough in real-time heuristic search performance was achieved through the use of case-based reasoning. In this framework, the agent optimally solves a set of problems and stores their solutions in a case base. Then, given any new problem, it seeks a similar case in the case base and uses its solution as an aid to solve the problem at hand. A number of ad hoc approaches to the case base formation problem have been proposed and empirically shown to perform well. In this paper, we investigate a theoretically driven approach to solving the problem. We mathematically relate properties of a case base to the suboptimality of the solutions it produces and subsequently develop an algorithm that addresses these properties directly. An empirical evaluation shows our new algorithm outperforms the existing state of the art on contemporary video-game pathfinding benchmarks.

Real-time heuristic search is suitable for time-sensitive pathfinding and planning tasks when an AI-controlled non-playable character must interleave its planning and plan execution. Since its inception in the early 90s, numerous real-time heuristic search algorithms have been proposed. Many of the algorithms also have control parameters leaving a practitioner with a bewildering array of choices. Recent work treated the task of algorithm and parameter selection as a search problem in itself. Such automatically found algorithms outperformed previously known manually designed algorithms on the standard video-game pathfinding benchmarks. In this paper we follow up by selecting an algorithm and parameters automatically per map. Our sampling-based approach is efficient on the standard video-game pathfinding benchmarks. We also apply the approach to per-problem algorithm selection and while it is effective there as well, it is not practical. We offer suggestions on making it so.

Video games often populate their in-game world with numerous ambient non-playable characters. Manually crafting interesting behaviors for such characters can be prohibitively expensive. As scripted AI gets re-used across multiple characters, they can appear overly similar, shallow and generally uninteresting for the player to interact with. In this paper we propose to evolve interesting behaviors in a simulated evolutionary environment. Since only some evolution runs may give rise to such behaviors, we propose to train deep neural networks to detect such behaviors. The paper presents work in progress in this direction.

Procedurally generating rich, naturally behaving AI-controlled video game characters is an important open problem. In this paper we focus on a particular aspect of non-playable character (NPC) behavior, long favored by science-fiction writers. Specifically, we study the effects of self-knowledge on NPC behavior. To do so we adopt the well-known framework of agent-centered real-time heuristic search applied to the standard pathfinding task on video-game maps. Such search agents normally use a heuristic function to guide them around a map to the goal state.

The symposium took place in July 2009 in Lake Arrowhead, California. Consequently, ARA techniques have been studied in various subfields in AI and related disciplines and have been used in various settings including automated reasoning, cognitive modeling, constraint programming, design, diagnosis, machine learning, model-based reasoning, planning, reasoning, scheduling, search, theorem proving, and intelligent tutoring. The considerable interest in ARA techniques and the great diversity of the researchers involved had led to work on ARA being presented at many different venues. Consequently, there was a need to have a single forum where researchers of different backgrounds and disciplines could discuss their work on ARA. As a result, the Symposium on Abstraction, Reformulation, and Approximation (SARA) was established in 1994 after a series of workshops in 1988, 1990, and 1992.

Real-time agent-centered heuristic search is a well-studied problem where an agent that can only reason locally about the world must travel to a goal location using bounded computation and memory at each step. Many algorithms have been proposed for this problem and theoretical results have also been derived for the worst-case performance with simple examples demonstrating worst-case performance in practice. Lower bounds, however, have not been widely studied. In this paper we study best-case performance more generally and derive theoretical lower bounds for reaching the goal using LRTA*, a canonical example of a real-time agent-centered heuristic search algorithm. The results show that, given some reasonable restrictions on the state space and the heuristic function, the number of steps an LRTA*-like algorithm requires to reach the goal will grow asymptotically faster than the state space, resulting in ``scrubbing'' where the agent repeatedly visits the same state. We then show that while the asymptotic analysis does not hold for more complex real-time search algorithms, experimental results suggest that it is still descriptive of practical performance.

Artificial Intelligence (AI) techniques have been widely used in video games to control non-playable characters. More recently, AI has been applied to automated story generation with the objective of managing the player's experience in an interactive narrative. Such AI experience managers can generate and adapt narrative dynamically, often in response to the player's in-game actions. We implement and evaluate a recently proposed AI experience manager, PACE, which predicts the player's emotional response to a narrative event and uses such predictions to shape the narrative to keep the player on an author-supplied target emotional curve.

Real-time heuristic search algorithms obey a constant limit on planning time per move. Agents using these algorithms can execute each move as it is computed, suggesting a strong potential for application to real-time video-game AI. Recently, a breakthrough in real-time heuristic search performance was achieved through the use of case-based reasoning. In this framework, the agent optimally solves a set of problems and stores their solutions in a case base. Then, given any new problem, it seeks a similar case in the case base and uses its solution as an aid to solve the problem at hand. A number of ad hoc approaches to the case base formation problem have been proposed and empirically shown to perform well. In this paper, we investigate a theoretically driven approach to solving the problem. We mathematically relate properties of a case base to the suboptimality of the solutions it produces and subsequently develop an algorithm that addresses these properties directly. An empirical evaluation shows our new algorithm outperforms the existing state of the art on contemporary video-game pathfinding benchmarks.