This paper asks how a player should exploit knowledge that their opponent in a repeated game is using a no-regret learning algorithm. Prior work has studied this question in Bayesian settings, such as when the learning player is a buyer and the rational player is a seller. This question extends the ideas to a non-Bayesian setting. In general, the rational player can guarantee the first-mover Stackelberg utility in the game. That is, being rational against a no-regret learner is worth at least as much as going first in a Stackelberg game.

Gomoku, also known as five in a row, is a classical board game, ideally suited for quickly testing novel Artificial Intelligence (AI) techniques. With the aim of facilitating a developer willing to write a new Gomoku player, in this report we present an analysis of the main game concepts and strategies, which is wider and deeper than existing ones. Moreover, after discussing the general structure of an artificial player, we present and analyse a strong Gomoku player, named Wine, the code of which is freely available on the Internet and which is an excelent example of how a modern player is organised.

Coordination games ([16]) are games in strategic form with several pure strategy Nash equilibria with the same or comparable payoffs for every player. In these games, all players have the mutual interest to select one of these equilibria. In pure coordination games ([16]), aka games of common payoffs ([17]), all players in the game receive the same payoffs and thus the players have fully aligned preferences to coordinate with each other in order to reach the best possible outcome for everyone. In this paper we study one-step pure winlose coordination games (WLC games) in which all payoffs are either 1 (i.e., win) or 0 (i.e., lose). Clearly, if players can communicate when playing a pure coordination game with at least one winning outcome, then they can simply agree on a winning strategy profile, so the game is trivialised.

Limited lookahead has been studied for decades in complete-information games. We initiate a new direction via two simultaneous deviation points: generalization to incomplete-information games and a game-theoretic approach. We study how one should act when facing an opponent whose lookahead is limited. We study this for opponents that differ based on their lookahead depth, based on whether they, too, have incomplete information, and based on how they break ties. We characterize the hardness of finding a Nash equilibrium or an optimal commitment strategy for either player, showing that in some of these variations the problem can be solved in polynomial time while in others it is PPAD-hard or NP-hard. We proceed to design algorithms for computing optimal commitment strategies---for when the opponent breaks ties favorably, according to a fixed rule, or adversarially. We then experimentally investigate the impact of limited lookahead. The limited-lookahead player often obtains the value of the game if she knows the expected values of nodes in the game tree for some equilibrium---but we prove this is not sufficient in general. Finally, we study the impact of noise in those estimates and different lookahead depths. This uncovers an incomplete-information game lookahead pathology.

Limited lookahead has been studied for decades in perfect-information games. This paper initiates a new direction via two simultaneous deviation points: generalization to imperfect-information games and a game-theoretic approach. The question of how one should act when facing an opponent whose lookahead is limited is studied along multiple axes: lookahead depth, whether the opponent(s), too, have imperfect information, and how they break ties. We characterize the hardness of finding a Nash equilibrium or an optimal commitment strategy for either player, showing that in some of these variations the problem can be solved in polynomial time while in others it is PPAD-hard or NP-hard. We proceed to design algorithms for computing optimal commitment strategies for when the opponent breaks ties 1) favorably, 2) according to a fixed rule, or 3) adversarially. The impact of limited lookahead is then investigated experimentally. The limited-lookahead player often obtains the value of the game if she knows the expected values of nodes in the game tree for some equilibrium, but we prove this is not sufficient in general. Finally, we study the impact of noise in those estimates and different lookahead depths. This uncovers a lookahead pathology.

Mutual exclusions provide useful information for learn- ing classes of concepts. We designed MuSweeper as a MineSweeper-like game to collect mutual exclusions from web users. Using the mechanism of an exten- sive game with Imperfect information, our experiments showed MuSweeper to collect mutual exclusions with high precision and efficiency.