Vision Transformers: Natural Language Processing (NLP) Increases Efficiency and Model Generality

There has been no shortage of developments vying for a share of your attention over the last year or so. However, if you regularly follow the state of machine learning research you may recall a loud contender for a share of your mind in OpenAI's GPT-3 and accompanying business strategy development from the group. GPT-3 is the latest and by far the largest in OpenAI's general purpose transformer lineage working on models for natural language processing. Of course, GPT-3 and GPTs may grab headlines, but it belongs to a much larger superfamily of transformer models, including a plethora of variants based on the Bidirectional Encoder Representations from Transformers (BERT) family originally created by Google, as well as other smaller families of models from Facebook and Microsoft. For an expansive but still not exhaustive overview of major NLP transformers, the leading resource is probably the Apache 2.0 licensed Hugging Face () library.