Mind the Gap: Machine Translation by Minimizing the Semantic Gap in Embedding Space

The conventional statistical machine translation (SMT) models, such as phrase-based models (Koehn et al. 2007), formal syntax-based models (Chiang 2007; Xiong, Liu, and Aiming at retaining the semantic meaning during the Lin 2006) and linguistically syntax-based models (Liu, Liu, translation process, we propose a Recursive Neural Network and Lin 2006; Huang, Knight, and Joshi 2006; Galley et al. (RNN) based translation model. Like the previous SMT 2006; Zhang et al. 2008), perform the decoding process and models, the RNN-based model induces the translation rules generate the translation result by compositing a set of translation from the bitexts. Unlike them, the RNN-based model learns rules which are associated with high probabilities. The how to represent each lexical translation rule with two compact probabilities of the translation rules (e.g. the phrasal translation semantic vectors, and learns how to perform decoding probabilities and the lexical weights in phrase-based using the merging type (swap or monotone) dependent recursive and formal syntax-based models) are all computed based on neural networks that attempt to find the best translation the cooccurrence statistics of the rule's source-and targetsides candidate having the minimal semantic gap with the source in the bilingual corpus.