Tainted Data Can Teach Algorithms the Wrong Lessons

An important leap for artificial intelligence in recent years is machines' ability to teach themselves, through endless practice, to solve problems, from mastering ancient board games to navigating busy roads. But a few subtle tweaks in the training regime can poison this "reinforcement learning," so that the resulting algorithm responds--like a sleeper agent--to a specified trigger by misbehaving in strange or harmful ways. "In essence, this type of back door gives the attacker some ability to directly control" the algorithm, says Wenchao Li, an assistant professor at Boston University who devised the attack with colleagues. Their recent paper is the latest in a growing body of evidence suggesting that AI programs can be sabotaged by the data used to train them. As companies, governments, and militaries rush to deploy AI, the potential for mischief could be serious.