Building Transformer Models with Attention Crash Course. Build a Neural Machine Translator in 12 Days - MachineLearningMastery.com Building Transformer Models with Attention Crash Course. Build a Neural Machine Translator in 12 Days - MachineLearningMastery.com

Moreover, when you look at the diagram of the transformer model and your implementation here, you should notice the diagram shows a softmax layer at the output, but we omitted that. The softmax is indeed added in this lesson. Do you see where is it? In the next lesson, you will train this compiled model, on 14 million parameters as we can see in the summary above. Training the transformer depends on everything you created in all previous lessons. Most importantly, the vectorizer and dataset from Lesson 03 must be saved as they will be reused in this and the next lessons. Running this script will take several hours, but once it is finished, you will have the model saved and the loss and accuracy plotted.