However, searching the adversarial state is challenging due to the large, discrete, and non-differentiable search space.

Prior work found that superhuman Go AIs like KataGo can be defeated by simple adversarial strategies. In this paper, we study if simple defenses can improve KataGo's worst-case performance. We test three natural defenses: adversarial training on hand-constructed positions, iterated adversarial training, and changing the network architecture. We find that some of these defenses are able to protect against previously discovered attacks. Unfortunately, we also find that none of these defenses are able to withstand adaptive attacks. In particular, we are able to train new adversaries that reliably defeat our defended agents by causing them to blunder in ways humans would not. Our results suggest that building robust AI systems is challenging even in narrow domains such as Go. For interactive examples of attacks and a link to our codebase, see https://goattack.far.ai/.

We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our adversaries do not win by playing Go well. Instead, they trick KataGo into making serious blunders. Our attack transfers zero-shot to other superhuman Go-playing AIs, and is comprehensible to the extent that human experts can implement it without algorithmic assistance to consistently beat superhuman AIs. The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack. Our results demonstrate that even superhuman AI systems may harbor surprising failure modes. Example games are available https://goattack.far.ai/.

In December, as AI chatbot ChatGPT awed the world with its human-like responses to questions, a major cheating scandal involving artificial intelligence was erupting in Asia. The Chunlan Cup, an international tournament boasting $200,000 in prize money for winning the ancient Chinese board game of Go, was embroiled in controversy following a semifinal match. In a David vs Goliath moment, a relative newcomer, Li Xuanhao of China, defeated the reigning world champion Shin Jin-seo of South Korea. On social media, Li's own teammate accused him of cheating using AI, which is commonly used during training but banned during competition. The controversy drew coverage from major newspapers, including Chinese state media.

AlphaZero is a self-play reinforcement learning algorithm that achieves superhuman play in chess, shogi, and Go via policy iteration. To be an effective policy improvement operator, AlphaZero's search requires accurate value estimates for the states appearing in its search tree. AlphaZero trains upon self-play matches beginning from the initial state of a game and only samples actions over the first few moves, limiting its exploration of states deeper in the game tree. We introduce Go-Exploit, a novel search control strategy for AlphaZero. Go-Exploit samples the start state of its self-play trajectories from an archive of states of interest. Beginning self-play trajectories from varied starting states enables Go-Exploit to more effectively explore the game tree and to learn a value function that generalizes better. Producing shorter self-play trajectories allows Go-Exploit to train upon more independent value targets, improving value training. Finally, the exploration inherent in Go-Exploit reduces its need for exploratory actions, enabling it to train under more exploitative policies. In the games of Connect Four and 9x9 Go, we show that Go-Exploit learns with a greater sample efficiency than standard AlphaZero, resulting in stronger performance against reference opponents and in head-to-head play. We also compare Go-Exploit to KataGo, a more sample efficient reimplementation of AlphaZero, and demonstrate that Go-Exploit has a more effective search control strategy. Furthermore, Go-Exploit's sample efficiency improves when KataGo's other innovations are incorporated.

Think that puny humans don't stand a chance when playing strategy games against an AI? You may have to think again. One person in the US beat an AI at the ancient game of Go by simply distracting it from the attack he was making, a tactic that would be unlikely to work on another meatbag. The player, Kellin Pelrine, is apparently not quite at the top of the amateur rankings for playing Go, but managed to best the AI in 14 out of 15 games, according to the Financial Times. Pelrine used tactics that involved distracting the algorithm with moves in other corners of the board while he worked to surround groups of his opponents' stones.

A strong amateur Go player has beat a highly-ranked AI system after exploiting a weakness discovered by a second computer, The Financial Times has reported. By exploiting the flaw, American player Kellin Pelrine defeated the KataGo system decisively, winning 14 of 15 games without further computer help. It's a rare Go win for humans since AlphaGo's milestone 2016 victory that helped pave the way for the current AI craze. It also shows that even the most advanced AI systems can have glaring blind spots. Pelrine's victory was made possible by a research firm called FAR AI, which developed a program to probe KataGo for weaknesses.

Players have often used KataGo to test their skills, train for other matches, and even analyze past games, yet in a study posted recently on the preprint server arXiv, researchers report that by using an adversarial policy--a kind of machine-learning algorithm built to attack or learn weaknesses in other systems--they've been able to beat KataGo at its own game between 50 to 99 percent of the time, depending on how much "thinking ahead" the AI does. "KataGo is able to recognize that passing would result in a forced win by our adversary, but given a low tree-search budget it does not have the foresight to avoid this," co-author Tony Wang, a Ph.D. student at MIT said of the study on the site LessWrong, an online community dedicated to "causing safe and beneficial AI."

KataGo's world-class AI learned Go by playing millions of games against itself, but that still isn't enough experience to cover every possible scenario, allowing for vulnerabilities from unexpected behavior. In the world of deep-learning artificial intelligence (AI), the ancient board game Go looms large. Until 2016, the best human Go player could still defeat the strongest Go-playing AI. That changed with DeepMind's AlphaGo, which used deep-learning neural networks to teach itself the game at a level humans cannot match. More recently, KataGo has become popular as an open source Go-playing AI that can beat top-ranking human Go players.