Intelligent Heuristics for the Game Isolation using AI and Minimax


How do you create an intelligent player for a game? Artificial intelligence offers a variety of ways to program intelligence into computer opponents. In this article, we'll show how it works, using intelligent heuristics and a web-based game that you can try yourself. Artificial intelligence is becoming an increasingly important topic in the field of computer science. While advancements in machine learning continue to break records in areas including image recognition, voice recognition, translation, and natural language processing, many additional branches of AI continue to advance as well. One of the earliest applications of AI is in the area of game development. Specifically, artificial intelligence is often used to create opponent players in games. Early forms of AI players in games often consisted of traditional board games, such as chess, checkers, backgammon, and tic-tac-toe. Games of this type provide a fully observable and deterministic view at any point in the state of the game. This allows an AI player the ability to analyze all possible moves from both the human player and the AI player itself, thus determining the best likely move to take at any given time. AI players in video games have since expanded to a much broader range of gaming categories, where the best move or course of action is not always crystal clear. These include games that often utilize random events or actions, in addition to hidden views of the game or of the opponent's actions.

Theoretical Analysis of Adversarial Learning: A Minimax Approach

Neural Information Processing Systems

In this paper, we propose a general theoretical method for analyzing the risk bound in the presence of adversaries. Specifically, we try to fit the adversarial learning problem into the minimax framework. We first show that the original adversarial learning problem can be transformed into a minimax statistical learning problem by introducing a transport map between distributions. Then, we prove a new risk bound for this minimax problem in terms of covering numbers under a weak version of Lipschitz condition. Our method can be applied to multi-class classification and popular loss functions including the hinge loss and ramp loss.

Exact Combinatorial Optimization with Graph Convolutional Neural Networks

Neural Information Processing Systems

Combinatorial optimization problems are typically tackled by the branch-and-bound paradigm. We propose a new graph convolutional neural network model for learning branch-and-bound variable selection policies, which leverages the natural variable-constraint bipartite graph representation of mixed-integer linear programs. We train our model via imitation learning from the strong branching expert rule, and demonstrate on a series of hard problems that our approach produces policies that improve upon state-of-the-art machine-learning methods for branching and generalize to instances significantly larger than seen during training. Moreover, we improve for the first time over expert-designed branching rules implemented in a state-of-the-art solver on large problems. Papers published at the Neural Information Processing Systems Conference.

Learning Compositional Neural Programs with Recursive Tree Search and Planning

Neural Information Processing Systems

We propose a novel reinforcement learning algorithm, AlphaNPI, that incorpo- rates the strengths of Neural Programmer-Interpreters (NPI) and AlphaZero. NPI contributes structural biases in the form of modularity, hierarchy and recursion, which are helpful to reduce sample complexity, improve generalization and in- crease interpretability. AlphaZero contributes powerful neural network guided search algorithms, which we augment with recursion. AlphaNPI only assumes a hierarchical program specification with sparse rewards: 1 when the program execution satisfies the specification, and 0 otherwise. This specification enables us to overcome the need for strong supervision in the form of execution traces and consequently train NPI models effectively with reinforcement learning.

Towards modular and programmable architecture search

Neural Information Processing Systems

Neural architecture search methods are able to find high performance deep learning architectures with minimal effort from an expert. However, current systems focus on specific use-cases (e.g. Hyperparameter optimization systems are general-purpose but lack the constructs needed for easy application to architecture search. In this work, we propose a formal language for encoding search spaces over general computational graphs. The language constructs allow us to write modular, composable, and reusable search space encodings and to reason about search space design.

Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game

Neural Information Processing Systems

Self-supervised (SS) learning is a powerful approach for representation learning using unlabeled data. Recently, it has been applied to Generative Adversarial Networks (GAN) training. Specifically, SS tasks were proposed to address the catastrophic forgetting issue in the GAN discriminator. In this work, we perform an in-depth analysis to understand how SS tasks interact with learning of generator. From the analysis, we identify issues of SS tasks which allow a severely mode-collapsed generator to excel the SS tasks.

Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning

Neural Information Processing Systems

Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-box functions. In this work, we introduce a method to automatically design the BO search space by relying on evaluations of previous black-box functions. We depart from the common practice of defining a set of arbitrary search ranges a priori by considering search space geometries that are learnt from historical data. This simple, yet effective strategy can be used to endow many existing BO methods with transfer learning properties.

Efficient Algorithms for Smooth Minimax Optimization

Neural Information Processing Systems

In terms of $g(\cdot,y)$, we consider two settings -- strongly convex and nonconvex -- and improve upon the best known rates in both. For strongly-convex $g(\cdot, y),\ \forall y$, we propose a new direct optimal algorithm combining Mirror-Prox and Nesterov's AGD, and show that it can find global optimum in $\widetilde{O}\left(1/k 2 \right)$ iterations, improving over current state-of-the-art rate of $O(1/k)$. We use this result along with an inexact proximal point method to provide $\widetilde{O}\left(1/k {1/3} \right)$ rate for finding stationary points in the nonconvex setting where $g(\cdot, y)$ can be nonconvex. This improves over current best-known rate of $O(1/k {1/5})$. Papers published at the Neural Information Processing Systems Conference.

Counting the Optimal Solutions in Graphical Models

Neural Information Processing Systems

We introduce #opt, a new inference task for graphical models which calls for counting the number of optimal solutions of the model. We describe a novel variable elimination based approach for solving this task, as well as a depth-first branch and bound algorithm that traverses the AND/OR search space of the model. The key feature of the proposed algorithms is that their complexity is exponential in the induced width of the model only. It does not depend on the actual number of optimal solutions. Our empirical evaluation on various benchmarks demonstrates the effectiveness of the proposed algorithms compared with existing depth-first and best-first search based approaches that enumerate explicitly the optimal solutions.

Improved Regret Bounds for Bandit Combinatorial Optimization

Neural Information Processing Systems

In this paper, we aim to reveal the property, which makes the bandit combinatorial optimization hard. This lower bound was achieved by considering a continuous strongly-correlated distribution of losses. Our main contribution is that we managed to improve this bound by $\Omega( \sqrt{d k 3 T})$ through applying a factor of $\sqrt{\log T}$, which can be done by means of strongly-correlated losses with \textit{binary} values. The bound derives better regret bounds for three specific examples of the bandit combinatorial optimization: the multitask bandit, the bandit ranking and the multiple-play bandit. In particular, the bound obtained for the bandit ranking in the present study addresses an open problem raised in \citep{cohen2017tight}.