Monte-Carlo Tree Search (MCTS) is an online planning algorithm that combines the ideas of best-first tree search and Monte-Carlo evaluation. Since MCTS is based on sampling, it does not require a transition function in explicit form, but only a generative model of the domain. Because it grows a highly selective search tree guided by its samples, it can handle huge search spaces with large branching factors. By using Monte-Carlo playouts, MCTS can take long-term rewards into account even with distant horizons. Combined with multi-armed bandit algorithms to trade off exploration and exploitation, MCTS has been shown to guarantee asymptotic convergence to the optimal policy, while providing approximations when stopped at any time. The relatively new MCTS approach has started a revolution in computer Go. Furthermore, it has achieved considerable success in domains as diverse as the games of Hex, Amazons, LOA, and Ms. Pacman; in General Game Playing, planning, and optimization. Whereas the focus of previous MCTS research has been on the practical application, current research begins to address the problem of understanding the nature, the underlying principles, of MCTS. A careful understanding of MCTS will lead to more effective search algorithms. Hence, my two interrelated research questions are: How can we formulate models that increase our understanding of how MCTS works? and How can we use the developed understanding to create effective search algorithms? This research summary describes the first steps I undertook in these directions, as well as my plans for future work.

In this paper, we survey techniques used in intelligent tutoring systems (ITSs) to model student knowledge. The three techniques that we review in detail are knowledge tracing, performance factor analysis, and matrix factorization. We also briefly cover other techniques that have been used. This review is meant to be a repository of knowledge for those who want to integrate these techniques into serious games. It is also meant to increase awareness and interest as to the techniques available that can be integrated into serious games.

Artificial Intelligence (AI) and Personalization are both essential - How do we relate content (the factual knowledge aspects of all games, be they serious or entertainment contained, game mechanics) and context (experiences based. In this research the role of AI and Personalization is and activities) to pedagogical goals towards supporting however focused upon the context of Serious Games (SG) in pedagogically-driven design and development of SGs? particular. A concerted research direction is necessary in this From these two high-level questions we derived a more area so as to establish future benchmarks and metrics for the pragmatic approach to AI and Personalization based on: In effective use of AI and Personalization in serious games design what ways can personalization improve learning and adapt and will benefit relevant research communities in providing best to learner requirements?

In this paper, methods and models for the design of educational interventions and usable systems are presented and synthesized. The purpose is to suplliment the design process with educational considerations and discern design principles for the development of serious STEM games. This synthesis can contribute to the design of the next generation of technologically enhanced learning environments.

This paper presents a tool geared towards the collaboration of a human and an artificial designer for the creation of game content. The framework combines procedural content generation using stochastic search with user input in the form of an initial goal statement as well as preference of generated results. Feedback from industry experts in a pilot user experiment showcased the limitations of this approach and the protocol chosen for evaluating the authoring tool. The limitations are discussed with respect to the suitability of interactive evolution for creative design and the design of experimental protocols for evaluating authoring tools for games.

In recent years various games have been developed to generate useful data for scientific and commercial purposes. Current human computation games are tailored around a task they aim to solve, adding game mechanics to conceal monotonous workflows. These gamification approaches, although providing valuable gaming experience, do not cover the wide range of experiences seen in digital games today. This work presents a new use for design concepts for human computation games and an evaluation of player experiences.

Beyond the goal of refining system design to the needs and tastes of users, user evaluation of interactive music systems offers a method of examining the nature of musical creativity as understood by its human practitioners. In the case of improvising music systems, user study and evaluation of a system’s ability to improvise may be useful in the ethnomusicological study of musical interaction in contemporary improvised music. A survey of preliminary findings based on the interactions of an improvising system, Maxine, with several improvisers is discussed, with results suggesting methodological reconfigurations of the purpose and goals of evaluating of interactive musical metacreations.

Complex games such as RTS games are naturally formalized as Markov games. Given a Markov game, it is often possible to hand-code or learn a set of policies that capture the diversity of possible strategies. It is also often possible to hand-code or learn an abstract simulator of the game that can estimate the outcome of playing two strategies against one another from any state. We consider how to use such policy sets and simulators to make decisions in large Markov games. Prior work has considered the problem using an approach we call minimax policy switching. At each decision epoch, all policy pairs are simulated against each other from the current state, and the minimax policy is chosen and used to select actions until the next decision epoch. While intuitively appealing, we show that this switching policy can have arbitrarily poor worst case performance. In response, we describe a modified algorithm, monotone policy switching, whose worst case performance, under certain conditions, is provably no worse than the minimax fixed policy in the set. We evaluate these switching policies in both a simulated RTS game and the real game Wargus. The results show the effectiveness of policy switching when the simulator is accurate, and also highlight challenges in the face of inaccurate simulations.

This paper describes our ongoing effort to build an empathizing and adaptive storyteller system. The system under development aims to utilize emotional expressions generated from an avatar or a humanoid robot in addition to the listener’s responses which are monitored in real time, in order to deliver a story in an effective manner. We conducted a pilot study and the results were analyzed in two ways: first, through a survey questionnaire analysis based on the participant’s subjective ratings; second, through automated video analysis based on the participant’s emotional facial expression and eye blinking. The survey questionnaire results show that male participants have a tendency of more empathizing with a story character when a virtual storyteller is present, as compared to audio-only narration. The video analysis results show that the number of eye blinking of the participants is thought to be reciprocal to their attention.

The Intentional Fast-Forward (IFF) planner is an attempt to apply fast forward-chaining state-space search methods to intentional planning---planning such that every action is directed toward some character's goal. The IFF heuristic is based on Hoffmann's original Fast Forward heuristic (2001), which solves a simplified version of the problem and uses that solution as a guide for the real problem. IFF incorporates constraints imposed by intentional planning to narrow down the set of steps which can be taken next, and it identifies fruitless branches of the search space early.