The Power of Scale

On May 2020, OpenAI introduced GPT-3 which is the third iteration of GPT language generation model series. The model boasted of a capacity of 175 billion parameters, more than 10 times than any other language model created before it. And to say that the model was a significant improvement would be an understatement. It could write essays on any topic without any inconsistencies in the output text. When trained on code samples, it could generate small code snippets (which were actually functional!) by getting a description of the task in English by the user. A group of developers trained it to update financial statements on Microsoft Excel based on casual description of transactions.