Scientists can't replicate AI studies. That's bad news.

The field of artificial intelligence (AI) may soon have to face a ghost that's haunted many a scientific field lately: the specter of replication. For a research study to be considered scientifically robust, the scientific method says that it must be possible for other researchers to reproduce its results under the same conditions. Yet because most AI researchers don't publish the source code they use to create their algorithms, it's been largely impossible for researchers to do that. Science magazine reports that at a meeting of the Association for the Advancement of Artificial Intelligence (AAAI), computer scientist Odd Erik Gundersen shared a report that found only six percent of 400 algorithms presented at two AI conferences in the past few years included the algorithm's code. Only one in three shared the data they used to test their program, and just half shared a summary that described the algorithm with limited detail -- AKA "pseudocode." Gundersen says that a change is going to be necessary as the field grows.