The Magical Rationalism of Elon Musk and the Prophets of AI

One morning in the summer of 2015, I sat in a featureless office in Berkeley as a young computer programmer walked me through how he intended to save the world. The world needed saving, he insisted, not from climate change -- or from the rise of the far right, or the treacherous instability of global capitalism -- but from the advent of artificial superintelligence, which would almost certainly wipe humanity from the face of the earth unless certain preventative measures were put in place by a very small number of dedicated specialists such as himself, who alone understood the scale of the danger and the course of action necessary to protect against it. This intense and deeply serious young programmer was Nate Soares, the executive director of MIRI (Machine Intelligence Research Institute), a nonprofit organization dedicated to the safe -- which is to say, non-humanity-obliterating -- development of artificial intelligence. As I listened to him speak, and as I struggled (and failed) to follow the algebraic abstractions he was scrawling on a whiteboard in illustration of his preferred doomsday scenario, I was suddenly hit by the full force of a paradox: The austere and inflexible rationalism of this man's worldview had led him into a grand and methodically reasoned absurdity. In researching and reporting my book, To Be a Machine, I had spent much of the previous 18 months among the adherents of the transhumanist movement, a broad church comprising life-extension advocates, cryonicists, would-be cyborgs, Silicon Valley tech entrepreneurs, neuroscientists looking to convert the human brain into code, and so forth -- all of whom were entirely convinced that science and technology would allow us to transcend the human condition.