Illustrated Guide to Transformer

For example, in machine translation, the input is an English sentence, and the output is the French translation. The Encoder will unroll each word in sequence and forms a fixed-length vector representation of the input English sentence. Then, the Decode will take the fixed-length vector representation as input, and produce each French word one after another, forming the translated English sentence. However, RNN models have some problems, they are slow to train, and they can't deal with long sequences. The input data needs to be processed sequentially one after the other.