In brief OpenAI released a new language model named ChatGPT this week, which is designed to mimic human conversations. The model is based on the company's latest text-generation GPT-3.5 system released earlier this year. ChatGPT is more conversational than previous versions. It can ask users follow-up questions and refrain from responding to inappropriate inputs instead of just generating text. Some examples show ChatGPT won't provide dangerous advice when prompted and can try to correct wrong statements.

Yet to navigate our unpredictable world, it needs to learn to make choices with imperfect information--as we do every single day. DeepMind just took a stab at solving this conundrum. The trick was to interweave game theory into an algorithmic strategy loosely based on the human brain called deep reinforcement learning. The result, DeepNash, toppled human experts in a highly strategic board game called Stratego. A notoriously difficult game for AI, Stratego requires multiple strengths of human wit: long-term thinking, bluffing, and strategizing, all without knowing your opponent's pieces on the board.

Game-playing artificial intelligence (AI) systems have advanced to a new frontier. Stratego, the classic board game that's more complex than chess and Go, and craftier than poker, has now been mastered. Published in Science, we present DeepNash, an AI agent that learned the game from scratch to a human expert level by playing against itself. DeepNash uses a novel approach, based on game theory and model-free deep reinforcement learning. Its play style converges to a Nash equilibrium, which means its play is very hard for an opponent to exploit.

While artificial intelligence long ago surpassed human capability in Chess, and more recently Go -- and let us not forget Doom -- other more complex board games still present a challenge to computer systems. Until very recently, Stratego and Diplomacy were two of those games, but now AI has become table-flipping good at the former and passably human at the latter. On the surface, you might think that it's just because these games require a certain level of long-term planning and strategy. But so do Go and Chess, just in a different way. The crucial difference is actually that Stratego and Diplomacy are games of strategy based on imperfect information.

An AI can defeat expert human players in the board game Stratego, which has more possible game scenarios than chess, Go or poker. The AI developed by the UK-based company DeepMind became one of the top-ranked online players of the Napoleonic-themed board game Stratego by learning to bluff with weaker pieces and sacrifice important pieces for the sake of victory. "To us the most surprising behaviour was [the AI's] ability to sacrifice valuable pieces to gain information about the opponent's set-up and strategy," says Julien Perolat at DeepMind. The game of Stratego involves two players trying to capture the opponent's flag hidden among an array of 40 game pieces. Most pieces consist of soldiers numbered from one to 10, with the higher-ranked soldiers defeating lower-ranked soldiers during encounters on the board. But players cannot see the identities of opponent game pieces unless two pieces from opposing armies encounter one another – unlike games such as chess or Go where both players can see everything.

Deepmind has been the pioneer in making AI models that have the capability to mimic a human's cognitive ability to play games. Games are a common testbed to assess a model's ability. After mastering games like Go, Chess and Checkers, Deepmind has launched DeepNash, an AI model that can play Stratego at an expert level. Mastering a game like'Stratego' is a significant achievement for AI research because it presents a challenging benchmark for learning strategic interactions at a massive scale. Stratego's complexity is based on two key aspects. Firstly, there are 10535 possible states in the game, which is exponentially larger than Texas hold'em poker(10164 states) and Go(10360 states).

We introduce DeepNash, an autonomous agent capable of learning to play the imperfect information game Stratego from scratch, up to a human expert level. Stratego is one of the few iconic board games that Artificial Intelligence (AI) has not yet mastered. This popular game has an enormous game tree on the order of $10^{535}$ nodes, i.e., $10^{175}$ times larger than that of Go. It has the additional complexity of requiring decision-making under imperfect information, similar to Texas hold'em poker, which has a significantly smaller game tree (on the order of $10^{164}$ nodes). Decisions in Stratego are made over a large number of discrete actions with no obvious link between action and outcome. Episodes are long, with often hundreds of moves before a player wins, and situations in Stratego can not easily be broken down into manageably-sized sub-problems as in poker. For these reasons, Stratego has been a grand challenge for the field of AI for decades, and existing AI methods barely reach an amateur level of play. DeepNash uses a game-theoretic, model-free deep reinforcement learning method, without search, that learns to master Stratego via self-play. The Regularised Nash Dynamics (R-NaD) algorithm, a key component of DeepNash, converges to an approximate Nash equilibrium, instead of 'cycling' around it, by directly modifying the underlying multi-agent learning dynamics. DeepNash beats existing state-of-the-art AI methods in Stratego and achieved a yearly (2022) and all-time top-3 rank on the Gravon games platform, competing with human expert players.