Implementing the Transformer Decoder From Scratch in TensorFlow and Keras

There are many similarities between the Transformer encoder and decoder, such as in their implementation of multi-head attention, layer normalization and a fully connected feed-forward network as their final sub-layer. Having implemented the Transformer encoder, we will now proceed to apply our knowledge in implementing the Transformer decoder, as a further step towards implementing the complete Transformer model. Our end goal remains the application of the complete model to Natural Language Processing (NLP). In this tutorial, you will discover how to implement the Transformer decoder from scratch in TensorFlow and Keras. Implementing the Transformer Decoder From Scratch in TensorFlow and Keras Photo by François Kaiser, some rights reserved.