Time to Fold, Humans: Poker-Playing AI Beats Pros at Texas Hold'em


It is no mystery why poker is such a popular pastime: the dynamic card game produces drama in spades as players are locked in a complicated tango of acting and reacting that becomes increasingly tense with each escalating bet. The same elements that make poker so entertaining have also created a complex problem for artificial intelligence (AI). A study published today in Science describes an AI system called DeepStack that recently defeated professional human players in heads-up, no-limit Texas hold'em poker, an achievement that represents a leap forward in the types of problems AI systems can solve. DeepStack, developed by researchers at the University of Alberta, relies on the use of artificial neural networks that researchers trained ahead of time to develop poker intuition. During play, DeepStack uses its poker smarts to break down a complicated game into smaller, more manageable pieces that it can then work through on the fly.

On Monte Carlo Tree Search and Reinforcement Learning

Journal of Artificial Intelligence Research

Fuelled by successes in Computer Go, Monte Carlo tree search (MCTS) has achieved widespread adoption within the games community. Its links to traditional reinforcement learning (RL) methods have been outlined in the past; however, the use of RL techniques within tree search has not been thoroughly studied yet. In this paper we re-examine in depth this close relation between the two fields; our goal is to improve the cross-awareness between the two communities. We show that a straightforward adaptation of RL semantics within tree search can lead to a wealth of new algorithms, for which the traditional MCTS is only one of the variants. We confirm that planning methods inspired by RL in conjunction with online search demonstrate encouraging results on several classic board games and in arcade video game competitions, where our algorithm recently ranked first. Our study promotes a unified view of learning, planning, and search.

AI Game-Playing Techniques

AI Magazine

In conjunction with the Association for the Advancement of Artificial Intelligence's Hall of Champions exhibit, the Innovative Applications of Artificial Intelligence held a panel discussion entitled "AI Game-Playing Techniques: Are They Useful for Anything Other Than Games?" This article summarizes the panelists' comments about whether ideas and techniques from AI game playing are useful elsewhere and what kinds of game might be suitable as "challenge problems" for future research.

The Hanabi Challenge: A New Frontier for AI Research

arXiv.org Machine Learning

From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay and imperfect information in a two to five player setting. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques capable of imbuing artificial agents with such theory of mind will not only be crucial for their success in Hanabi, but also in broader collaborative efforts, and especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.

Bested by AI: What Happens When AI Wins?


A few months ago, I sent my dad the article 20 Top Lawyers Were Beaten by Legal AI in a Controlled Study, which (as the title suggests) discusses a study on how AI can be applied to the field of law, and how it performs against professional lawers. An implication of this article is the potential to replace lawyers with AI for many common legal needs, such as contract review or writing wills. It's an interesting article and application of AI, which I spend a lot of time thinking about. It might seem pretty innocent that I shared it with my dad, and it would be, except that my dad is a lawyer. Yes, I was kind of trying to get a rise out of him (it's all affectionate, I promise).