We're Not Ready For Superintelligence

The problem with the world today isn't that too many people are afraid--it's that too many people are afraid of the wrong things. Consider this: what scares you more, that your life could end because of a terrorist attack or because you get crushed to death under a large piece of furniture? Despite a media environment in which the threat of terrorism is omnipresent and the threat of furniture nonexistent, your gravestone is actually more likely to say, "Died under a couch recently bought from Ikea" than "Perished in a terrorist attack." In fact, asteroids are more likely to kill the average person than lightning strikes, and lightning strikes are more dangerous than terrorism. The point is that, as I've written elsewhere, our intuitions often fail to track the actual risks around us. We dismiss many of the most likely threats while obsessing over improbable events.