FRAGE, we achieve higher performance than the baselines in all tasks.

We introduce Generative Neural Machine Translation (GNMT), a latent variable architecture which is designed to model the semantics of the source and target sentences. We modify an encoder-decoder translation model by adding a latent variable as a language agnostic representation which is encouraged to learn the meaning of the sentence. GNMT achieves competitive BLEU scores on pure translation tasks, and is superior when there are missing words in the source sentence. We augment the model to facilitate multilingual translation and semi-supervised learning without adding parameters. This framework significantly reduces over-fitting when there is limited paired data available, and is effective for translating between pairs of languages not seen during training.

Neural Information Processing Systems http://nips.cc/

Recent progress has demonstrated that such meta-learning methods may exceed scalable human-invented architectures on image classification tasks.

To overcome this limitation, we propose a novel blockwise parallel decoding scheme in which we make predictions for multiple time steps in parallel then back off to the longest prefix validated by a scoring model.

Figure 1: Sketch of variational attention applied to machine translation.

In this paper, we provide a theoretical understanding of word embedding and its dimensionality. Motivated by the unitary-invariance of word embedding, we propose the Pairwise Inner Product (PIP) loss, a novel metric on the dissimilarity between word embeddings. Using techniques from matrix perturbation theory, we reveal a fundamental bias-variance trade-off in dimensionality selection for word embeddings. This bias-variance trade-off sheds light on many empirical observations which were previously unexplained, for example the existence of an optimal dimensionality. Moreover, new insights and discoveries, like when and how word embeddings are robust to over-fitting, are revealed. By optimizing over the bias-variance trade-off of the PIP loss, we can explicitly answer the open question of dimensionality selection for word embedding.

As a case study, we present such a system that learns to parse Newtonian physics problems in textbooks.

Machine comprehension (MC) has gained significant popularity over the past few years and it is a coveted goal in the field of natural language understanding.

Neural Information Processing Systems http://nips.cc/