A Deep Dive Into Machine Translation

The training sentences vary in length from a single word all the way up to very long run-on sentences. TensorFlow does not currently support variable length sequences in RNNs, so this is a problem. In order to deal with these variable length inputs and outputs, the data could be padded to be the same length as the longest input and output, but this would create a lot of pad symbols in our training data and would probably mess with the output. We could also create a model for each possible length of input and output, but that would create a very large number of modules and require a much larger amount of training data. The TensorFlow framework recommends solving this problem through bucketing.