Monte-Carlo Tree Search (MCTS) is an online planning algorithm that combines the ideas of best-first tree search and Monte-Carlo evaluation. Since MCTS is based on sampling, it does not require a transition function in explicit form, but only a generative model of the domain. Because it grows a highly selective search tree guided by its samples, it can handle huge search spaces with large branching factors. By using Monte-Carlo playouts, MCTS can take long-term rewards into account even with distant horizons. Combined with multi-armed bandit algorithms to trade off exploration and exploitation, MCTS has been shown to guarantee asymptotic convergence to the optimal policy, while providing approximations when stopped at any time. The relatively new MCTS approach has started a revolution in computer Go. Furthermore, it has achieved considerable success in domains as diverse as the games of Hex, Amazons, LOA, and Ms. Pacman; in General Game Playing, planning, and optimization. Whereas the focus of previous MCTS research has been on the practical application, current research begins to address the problem of understanding the nature, the underlying principles, of MCTS. A careful understanding of MCTS will lead to more effective search algorithms. Hence, my two interrelated research questions are: How can we formulate models that increase our understanding of how MCTS works? and How can we use the developed understanding to create effective search algorithms? This research summary describes the first steps I undertook in these directions, as well as my plans for future work.

We examine the use of constraint propagation for populating indoor game levels with enemies and other objects.  We introduce a notion of path constraints , which bound some function over the possible paths a player might take, and show how to efficiently place objects while guaranteeing path constraints.  This allows the system to guarantee that power-ups are balanced to the number of enemies occurring in the level, that they’re placed early enough to be useful, that keys are not hidden behind the doors they are intended to unlock, and so on. We describe a constraint solver based on interval methods that allows natural processing of numeric constraints and show that it is efficient enough to be used even on very low-end platforms.

We present CLASS Q-L (for: class Q-learning) an application of the Q-learning reinforcement learning algorithm to play complete Wargus games. Wargus is a real-time strategy game where players control armies consisting of units of different classes (e.g., archers, knights). CLASS Q-L uses a single table for each class  of unit so that each unit is controlled and updates its class’ Q-table. This enables rapid learning as in Wargus there are many units of the same class. We present initial results of CLASS Q-L against a variety of opponents.

The field of game AI is largely industry driven, lacking an agreed upon formalism for AI representation. Ad-hoc scripting languages, simple finite state machines, behaviour trees, and planners are employed, but not in a fashion adhering to any standard. As a result, reuse is sparse between games and formal analysis techniques are undeveloped. As research for a Ph.D. thesis, we propose to show that a layered Statechart-based AI is a suitable formalism for Game AI, enabling the use of model-driven development techniques such as reuse and high-level analysis including model-checking. The fundamentally modular nature of this approach leads naturally to reuse as a fundamental component of the design process. Supported by a clearly defined formalism, useful behavioural analyses become possible, such as testing reactions to various inputs at design time. We also explore transformations at the modelling level to enable procedural generation, allowing rapid deployment of varying AIs. Additionally, such a model allows for the generation of efficient code that can be directly inserted into games. Tool support for reuse, generation, and analysis will be developed, then employed in creating an industrial scale AI, proving that this formalism is appropriate for industrial use.

Imagine that a budding composer suffers from writer's block partway through devising a melody. A system called FreshJam is demonstrated, which offers a solution to this problem in the form of an interactive composition assistant; an algorithm that analyzes the notes composed so far, makes a comparison with an indexed corpus of existing music, and suggests a possible next note by choosing randomly among continuations of matched melody fragments. We provide a demonstration of FreshJam as an aid in stylistic composition, and of its potential to be more iterative than existing composition assistants such as PG Music's Band in a Box or Microsoft's Songsmith.

Around the same time, a system called Player-Specific From Shakespeare's "Romeo and Juliet" to George Lucas' Stories via Automatically Generated Events (PaSSAGE) "Star Wars" to BioWare's "Jade Empire" to campfire stories (Thue et al. 2007) was proposed, which used AI techniques to baseball commentary, story-telling is a fundamental to model the player as he/she experiences a narrative-rich part of entertainment. A strong narrative resonates with our video game. Such a continuously updated player model was minds, hearts and souls and keeps us engaged. We remember used to dynamically adapt the story, tailoring it to the current the stories of our childhood and retell them to our own player. Unlike, ASD, PaSSAGE did not have any automation children. Story-telling has delighted and saddened the human at the design stage and relied on a human designer to race since the beginning of time and shows no signs of foresee all possible ways of a player breaking the story and slowing down. But can it be improved with technology?

This paper describes a prototype of a two-player collaborative 2D puzzle game, designed to elicit task-oriented situated dialog. In this game players use a text-based chat to coordinate their actions in pushing a ball through a maze of obstacles. The game will be used to collect corpora of human-human interactions in this environment. The data will be used to study how language with actions are interleaved and influence each other in situated dialog. The ultimate goal is to build a computational model of these behaviors.

Adapting game content to a particular player's needs and expertise constitutes an important aspect in game design. Most research in this direction has focused on adapting game difficultyto keep the player engaged in the game. Dynamic difficulty adjustment, however, focuses on one aspect of the gameplay experience by adjusting the content to increase ordecrease perceived challenge. In this paper, we introduce a method for automatic level generation for the platform game Super Mario Bros using grammatical evolution. The grammatical evolution-based level generator is used to generate player-adapted content by employing an adaptation mechanism as a fitness function in grammatical evolution to optimizethe player experience of three emotional states: engagement, frustration and challenge. The fitness functions used are models of player experience constructed in our previous work from crowd-sourced gameplay data collected from over 1500 game sessions.

Quantum mechanical systems exist as superpositions of complementary states that collapse to classical, concrete states upon becoming entangled with the measurement apparatus of observer-participants. A musical composition and its performance constitute a quantum system. Historically, conventional musical notation has presented the appearance of a composition as a deterministic, concrete entity, with interpretation approached as an extrinsic act. This historical perspective inhabits a subspace of the available quantum space. A quantum musical system unifies the composition, instruments, situated performance and perception as a superposition of musical events that collapses to concrete musical events via the interactions and perceptions of performers and audience. A composer captures superposed musical events via implicit or explicit conditional event probabilities, and human and/or machine performers create music by collapsing interrelated probabilities to zeros and ones via observer-participancy.

Beyond the goal of refining system design to the needs and tastes of users, user evaluation of interactive music systems offers a method of examining the nature of musical creativity as understood by its human practitioners. In the case of improvising music systems, user study and evaluation of a system’s ability to improvise may be useful in the ethnomusicological study of musical interaction in contemporary improvised music. A survey of preliminary findings based on the interactions of an improvising system, Maxine, with several improvisers is discussed, with results suggesting methodological reconfigurations of the purpose and goals of evaluating of interactive musical metacreations.