The 'Don't Look Up' Thinking That Could Doom Us With AI

Many companies are working to build AGI (artificial general intelligence), defined as "AI that can learn and perform most intellectual tasks that human beings can, including AI development." Below we'll discuss why this may rapidly lead to superintelligence, defined as "general intelligence far beyond human level". I'm often told that AGI and superintelligence won't happen because it's impossible: human-level Intelligence is something mysterious that can only exist in brains. Such carbon chauvinism ignores a core insight from the AI revolution: that intelligence is all about information processing, and it doesn't matter whether the information is processed by carbon atoms in brains or by silicon atoms in computers. AI has been relentlessly overtaking humans on task after task, and I invite carbon chauvinists to stop moving the goal posts and publicly predict which tasks AI will never be able to do.