A brief timeline of NLP from Bag of Words to the Transformer family

Bag of Words (BOW) [1954]: count the occurrences of each word in the documents and use them as features. TF-IDF [1972]: the BOW scores are modified so that rare words have high scores and common words have low scores. Word2Vec [2013]: each word is mapped to a high-dimensional vector called word embedding, which captures its semantic. Word embeddings are learned by a neural network looking for word correlations on a large corpus. RNN [1986]: RNNs compute document embeddings leveraging word context in sentences, which was not possible with word embeddings alone.