Transformers: Age of Attention

This is the forth and final post in our series of blog posts focusing on the field of Natural Language Processing! In our first post, we saw that the application of neural networks for building language models was a major turning point in the NLP timeline, and in our second post we explored the significance of Word Embeddings in advancing the field. In our third post, we described the language model and how to build your own language-generating model in Keras! We are finally ready to tackle sequential processing, attention, and the Transformer! In their highly-memorable paper titled "Attention Is All You Need", Google Brain researchers introduced the Transformer, a new type of encoder-decoder model that relies solely on attention for sequence-to-sequence modelling.