The (Un)ethical Story of GPT-3: OpenAI's Million Dollar Model

Back on October 12, 2019, the world witnessed a previously unimaginable accomplishment- the first sub-two-hour marathon was run in an incredible time of 1:59:40 by Kenyan native Eliud Kipchoge. He would later say in regards to the amazing achievement that he "expected more people all over the world to run under 2 hours after today" [1]. While Kipchoge set new records in long distance running, across the world a team of natural language processing (NLP) experts at OpenAI, the Elon Musk-backed AI firm, published a new transformer-based language model with 1.5 billion parameters that achieved previously unthinkable performance in nearly every language task it faced [2]. The main takeaway from the paper by many experts was that bigger is better-the intelligence of transformer models can dramatically increase with the scale of parameters. In March of 2020, this theory gained support with OpenAI's release of version three of the model or GPT-3 which encapsulates a staggering 175 billion parameters and achieved even more remarkable performance than version 2, despite sharing, quite literally, the same architecture [3].