[D] Quasi-RNN NMT Decoder evaluation time • r/MachineLearning

Convolution operation helps in leaning the context much faster than the LSTMs. The encoder can be parallelized using the Convolution, however, I am confused with parallelization of the decoder. During training, when we know the output translated sentence, we can provide the decoder the output sentence as the input by shifting it one time to the right. However, during testing, we have to run the decoder n times to extract n words of the output sentence, using the predicted word in the current time step as the input to the decoder in the next timestep. Using a decoder with LSTM / RNN layers would have increased the per layer execution time complexity, where a convolutional decoder can execute each layer parallel, but LSTM decoder would have still run 1 time compared to n times of convolutional decoder.