Training Neural Machine Translation (NMT) Models using Tensor Train Decomposition on TensorFlow (T3F)

Neural Machine Translation (NMT) is a deep learning model that prov ides a robust method for machine translation using recurrent neural ne tworks (RNNs). Originally proposed in [1], NMT relies primarily on an encoder-decoder ar chi-tecture that provides increased fluency over phrase-based sys tems. This was implemented successfully in [2] for fast, accurate use on very large datasets. However, it has been suggested that there is significant redundan cy in the current method of neural network parametrization [3], presenting t he opportunity for significant speedup. Tensor Train (TT) decomposition [4] is a method by which large tenso rs can be approximated by the product of a'train' of smaller matrices (see Section 2.2). 1 TTdecomposition has been proposed as a method of speeding up an d reducing the memory usage of machine translation systems with dense weight matrices by reducing the number of parameters required to describe the sy stem [3].