If Planet Death Doesn't Get Us, an AI Superintelligence Most Certainly Will

Trying to comprehend the nature of an AI superintelligence is akin to trying to comprehend the mind of God. There's an element of the mystical in attempting to define the intentions of a being infinitely more intelligent and therefore infinitely more powerful than you--a being that may or may not even exist. There's even an actual religion, founded by a former Uber and Waymo self-driving engineer, dedicated to the worship of God based on artificial intelligence. But the existential fear of superintelligence isn't just spiritual vaporware, and we have a sense, at least, of how it might come to be. Researchers like Eliezer Yudkowsky suggest that the development of superintelligence could start with the creation of a "seed AI"--an artificial intelligence program that is able to achieve recursive self-improvement. We've already done that with narrow AI-- AlphaGo Zero started out knowing nothing about the game Go, but quickly managed to improve to a human level and then far beyond.