The Battle Over Books3 Could Change AI Forever

After OpenAI released GPT-3 in July 2020, independent artificial intelligence researcher Shawn Presser and a few of his fellow machine-learning enthusiasts set a challenge for themselves: Could they recreate it? "We were like, OK, there's actually not that much standing in the way of us doing this ourselves," Presser says. So what if OpenAI had deep pockets and a head start? That summer, they pored over papers about GPT-3, strategizing in marathon Discord chats about how to best approximate its training data sets. Presser honed in on the books they needed.