What is s driving the innovation in NLP and GPT-3?

Introduced in 2017, the Transformer is a deep learning model designed for NLP. Like recurrent neural networks (RNNs), Transformers handle sequential data. However, unlike RNNs, due to the attention mechanism, Transformers do not require that the data be processed in a sequential manner. This allows for much more parallelization in Transformers (in comparison to RNNs). In turn, parallelization during training allows for training on larger datasets.