In certain situations, the bots also showed remarkable speed, finding bugs far quicker than a human ever could. But at the same time, they proved that automated security is still very flawed. One bot quit working midway through the contest. Another patched a hole but, in the process, crippled the machine it was supposed to protect. All the gathered researchers agreed that these bots are still a very long way from grasping all the enormously complex bugs a human can.

February 18, 2017 --Observing lemurs in the jungles of Madagascar is no easy task. "We find the group," explains Stacey Tecot, an assistant professor at the University of Arizona's School of Anthropology, "and then we watch them for a little bit, we get our bearings ... and then we start to collect our data." Doing so is an all-day process of recording each individual, more or less continuously. But lemurs typically live in "troops" of up to 15 individuals. To get solid data, Dr. Tecot tells The Christian Science Monitor in a phone interview, "you really have to know that who you're watching is who you think you're watching."

During the contest, which played out over a matter of hours, one bot proved it could find and exploit a particularly subtle security hole similar to one that plagued the world's email systems a decade ago--the Crackaddr bug. Until yesterday, this seemed beyond the reach of anything other than a human. "That was astounding," said Mike Walker, the veteran white-hat hacker who oversaw the contest. "Anybody who does vulnerability research will find that surprising." In certain situations, the bots also showed remarkable speed, finding bugs far quicker than a human ever could.