AAAI Conferences


Modular Architecture for StarCraft II with Deep Reinforcement Learning

AAAI Conferences

We present a novel modular architecture for StarCraft II AI. The architecture splits responsibilities between multiple modules that each control one aspect of the game, such as build-order selection or tactics. A centralized scheduler reviews macros suggested by all modules and decides their order of execution. An updater keeps track of environment changes and instantiates macros into series of executable actions. Modules in this framework can be optimized independently or jointly via human design, planning, or reinforcement learning. We present the first result of applying deep reinforcement learning techniques to training two out of six modules of a modular agent with self-play, achieving 92% or 86% win rates against the "Harder" (level 5) built-in Blizzard bot in Zerg vs. Zerg matches, with or without fog-of-war.


A Hybrid Approach to Planning and Execution in Dynamic Environments Through Hierarchical Task Networks and Behavior Trees

AAAI Conferences

Intelligent autonomous agents that are acting in dynamic environmentsin real-time are often required to follow long-termstrategies while also remaining reactive and being able to actdeliberately. In order to create intelligent behaviors for videogame characters, there are two common approaches – plannersare used for long-term strategical planning, whereas BehaviorTrees allow for reactive acting. Although both methodologieshave their advantages, when used on their own, theyfail to fully achieve both requirements described above. Inthis work, we propose a hybrid approach combining a HierarchicalTask Network planner for high-level planning whiledelegating low-level decision making and acting to BehaviorTrees. Furthermore, we compare this approach with a pureplanner in a multi-agent environment.


CADI — A Conversational Assistive Design Interface for Discovering Pong Variants

AAAI Conferences

Mixed-initiative PCG systems provide a way to leverage the expressive power of algorithmic techniques for content generation in a manner that lowers the technical barrier for content creators. While these tools are a proof of concept of how PCG systems can aide aspiring designers reach their vision, there are issues pertaining capturing designer intent, and interface complexity. In this paper we introduce CADI (Conversational Assistive Design Interface) a mixed initiative PCG system for creating variations of the game Pong that utilizes natural language input through a natural language interface to explore the design space of Pong variations. We provide a motivation for the creation of CADI and discuss the implementation and design decisions taken to address the issues of designer intent and interface complexity in mixed-initiative PCG systems.


Modeling Player Experience with the N-Tuple Bandit Evolutionary Algorithm

AAAI Conferences

Automatic game design is an increasingly popular area of research that consists of devising systems that create content or complete games autonomously. The interest in such systems is two-fold: games can be highly stochastic environments that allow presenting this task as a complex optimization problem and automatic play-testing, becoming benchmarks to advance the state of the art on AI methods. In this paper, we propose a general approach that employs the N-Tuple Bandit Evolutionary Algorithm (NTBEA) to tune parameters of three different games of the General Video Game AI (GVGAI) framework. The objective is to adjust the game experience of the players so the distribution of score events through the game approximates certain pre-defined target curves. We report satisfactory results for different target score trends and games, paving the path for future research in the area of automatically tuning player experience.


Tracing Player Knowledge in a Parallel Programming Educational Game

AAAI Conferences

This paper focuses on tracing player knowledge in educational games. Specifically, given a set of concepts or skills required to master a game, the goal is to estimate the likelihood with which the current player has mastery of each of those concepts or skills. The main contribution of the paper is an approach that integrates machine learning and domain knowledge rules to find when the player applied a certain skill and either succeeded or failed. This is then given as input to a standard knowledge tracing module (such as those from Intelligent Tutoring Systems) to perform knowledge tracing. We evaluate our approach in the context of an educational game called Parallel to teach parallel and concurrent programming with data collected from real users, showing our approach can predict students skills with a low mean-squared error.


Exploring Gameplay With AI Agents

AAAI Conferences

The process of play testing a game is subjective, expensive and incomplete. In this paper, we present a play-testing approach that explores the game space with automated agents and collects data to answer questions posed by the designers. Rather than have agents interacting with an actual game client, this approach recreates the bare bone mechanics of the game as a separate system. Our agent is able to play in minutes what would take testers days of organic gameplay. The analysis of thousands of game simulations exposed imbalances in game actions, identified inconsequential rewards and evaluated the effectiveness of optional strategic choices. Our test case game, The Sims Mobile, was recently released and the findings shown here influenced design changes that resulted in improved player experience.


Like a DNA String: Sequence-Based Player Profiling in Tom Clancy’s The Division

AAAI Conferences

In this paper we present an approach to using sequence analysis to model player behavior. This approach is designed to work in game development contexts, integrating production teams and delivering profiles that inform game design. We demonstrate the method via a case study of the game T om Clancy’s The Division, which with its 20 million players represents a major current commercial title. The approach presented provides a mixed-methods framework, combining qualitative knowledge elicitation and workshops with large-scale telemetry analysis, using sequence mining and clustering to develop detailed player profiles showing the core game-play loops of The Division’s players.


A Monte Carlo Approach to Skill-Based Automated Playtesting

AAAI Conferences

In order to create well-crafted learning progressions, designers guide players as they present game skills and give ample time for the player to master those skills. However, analyzing the quality of learning progressions is challenging, especially during the design phase, as content is ever-changing. This research presents the application of Stratabots — automated player simulations based on models of players with varying sets of skills — to the human computation game Foldit. Stratabot performance analysis coupled with player data reveals a relatively smooth learning progression within tutorial levels, yet still shows evidence for improvement. Leveraging existing general gameplaying algorithms such as Monte Carlo Evaluation can reduce the development time of this approach to automated playtesting without losing predicitive power of the player model.


A Design Pattern Approach for Multi-Game Level Generation

AAAI Conferences

Existing approaches to multi-game level generation rely upon level structure to emerge organically via level fitness. In this paper, we present a method for generating levels for games in the GVGAI framework using a design pattern-based approach, where design patterns are derived from an analysis of the existing corpus of GVGAI game levels. We created two new generators: one constructive, and one search-based, and compared them to a prior existing search-based generator. Results show that our generator is comparable, even preferred, over the prior generator, especially among players with existing game experience. Our search-based generator also outperforms our constructive generator in terms of player preference.


Talin: A Framework for Dynamic Tutorials Based on the Skill Atoms Theory

AAAI Conferences

Most tutorials in video games do not consider the skill level of the player when deciding what information to present. This makes many tutorials either tedious for experienced players or not informative enough for players who are new to the given genre. With Talin, implemented as an asset in the Unity game engine, we make it possible to create a mastery model of an individual player's skill levels by operationalizing Dan Cook's skill atom theory. We propose that using this mastery model opens up a new design space when it comes to designing tutorials. We show an example tutorial implementation with Talin assembled using only graphical components provided by our framework, without the need of writing any code. The dynamic tutorial implementation results in the player receiving information only when they need it, whenever they need it. While the novice player is given all the information they need to learn the system, the expert player is not bogged down by tooltip pop-ups regarding mechanics they have already mastered.