Containment algorithms won't stop super-intelligent AI, scientists warn

A team of computer scientists has used theoretical calculations to argue that algorithms could not control a super-intelligent AI. Their study addresses what Oxford philosopher Nick Bostrom calls the control problem: how do we ensure super-intelligence machines act in our interests? The researchers conceived of a theoretical containment algorithm that would resolve this problem by simulating the AI's behavior, and halting the program if its actions became harmful. If you break the problem down to basic rules from theoretical computer science, it turns out that an algorithm that would command an AI not to destroy the world could inadvertently halt its own operations. If this happened, you would not know whether the containment algorithm is still analyzing the threat, or whether it has stopped to contain the harmful AI.