An unsophisticated translation of such a sentence will therefore not be a good translation. Again, contrary to Mr. Richensi opinion, I believe that the problem involved is serious. There is no simple procedure to find out which, and in what way, the words of the English language are context-dependent. And I don't think that the issue can be belittled for tae reason that contextdependent words do not occur in scientific discussions and writings. They might not be too abundant in ordinary scientific papers on matters physical or chemical, but there would surely be plenty of them in discussions of matters linguistic, for instance. This might be one reason why so far hardly anybody has tried to machine translate papers in linguistics. As soon as this is attempted, the seriousness of the problem will become immediately evident.

If ability to perform complex calculations were a sufficient criterion, then even a conventional digital computor could lay claim to more intelligence than any of usand perhaps we had better let it make away with the word and be done with it.

Networks as models of word storage: G. R. Kiss

Theorem Proving Vision Robotics Information Processing Psychology Learning and Inductive Inference Planning and Related Problem-solving Techniques A. Natural Language Processing Ovnrview The most common way that human beings communicate Is by speaking or writing In one of the "natural" languages, like English, French, or Chinese. Computer programming languages, on the other hand, seem awkward to humans. These "artificial" languages are designed to have a rigid format, or syntax, so that a computer program reading and compiling code written In an artificial language can understand what the programmer means. In addition to being structurally simpler than natural languages, the artificial languages can express easily only those concepts that are important In programming: "Do this then do that," "See it such and such Is true," etc. The things that can be expressed In a language are referred to as the semantics of the language. The research on understanding natural language described in this section of the Handbook is concerned with programs that deal with the full range of meaning of languages like English.

Cross-language learning allows us to use training data from one language to build models for a different language. Many approaches to bilingual learning require that we have word-level alignment of sentences from parallel corpora. In this work we explore the use of autoencoder-based methods for cross-language learning of vectorial word representations that are aligned between two languages, while not relying on word-level alignments. We show that by simply learning to reconstruct the bag-of-words representations of aligned sentences, within and between languages, we can in fact learn high-quality representations and do without word alignments. We empirically investigate the success of our approach on the problem of cross-language text classification, where a classifier trained on a given language (e.g., English) must learn to generalize to a different language (e.g., German). In experiments on 3 language pairs, we show that our approach achieves state-of-the-art performance, outperforming a method exploiting word alignments and a strong machine translation baseline.

We explore how to improve machine translation systems by adding more translation data in situations where we already have substantial resources. The main challenge is how to buck the trend of diminishing returns that is commonly encountered. We present an active learning-style data solicitation algorithm to meet this challenge. We test it, gathering annotations via Amazon Mechanical Turk, and find that we get an order of magnitude increase in performance rates of improvement.

Building machine translation (MT) test sets is a relatively expensive task. As MT becomes increasingly desired for more and more language pairs and more and more domains, it becomes necessary to build test sets for each case. In this paper, we investigate using Amazon's Mechanical Turk (MTurk) to make MT test sets cheaply. We find that MTurk can be used to make test sets much cheaper than professionally-produced test sets. More importantly, in experiments with multiple MT systems, we find that the MTurk-produced test sets yield essentially the same conclusions regarding system performance as the professionally-produced test sets yield.

On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Kyunghyun Cho Bart van Merri enboer Universit e de Montr eal Dzmitry Bahdanau Jacobs University, Germany Yoshua Bengio Universit e de Montr eal, CIFAR Senior Fellow Abstract Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder-Decoder and a newly proposed gated recursive con-volutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically. 1 Introduction A new approach for statistical machine translation based purely on neural networks has recently been proposed (Kalchbrenner and Blunsom, 2013; Sutskever et al., 2014). This new approach, which we refer to as neural machine translation, is inspired by the recent trend of deep representational learning. All the neural network models used in (Kalchbrenner and Blunsom, 2013; Sutskever et al., 2014; Cho et al., 2014) consist of an encoder and a decoder.

We describe a unified and coherent syntactic framework for supporting a semantically-informed syntactic approach to statistical machine translation. Semantically enriched syntactic tags assigned to the target-language training texts improved translation quality. The resulting system significantly outperformed a linguistically naive baseline model (Hiero), and reached the highest scores yet reported on the NIST 2009 Urdu-English translation task. This finding supports the hypothesis (posed by many researchers in the MT community, e.g., in DARPA GALE) that both syntactic and semantic information are critical for improving translation quality---and further demonstrates that large gains can be achieved for low-resource languages with different word order than English.

The conventional statistical machine translation (SMT) models, such as phrase-based models (Koehn et al. 2007), formal syntax-based models (Chiang 2007; Xiong, Liu, and Aiming at retaining the semantic meaning during the Lin 2006) and linguistically syntax-based models (Liu, Liu, translation process, we propose a Recursive Neural Network and Lin 2006; Huang, Knight, and Joshi 2006; Galley et al. (RNN) based translation model. Like the previous SMT 2006; Zhang et al. 2008), perform the decoding process and models, the RNN-based model induces the translation rules generate the translation result by compositing a set of translation from the bitexts. Unlike them, the RNN-based model learns rules which are associated with high probabilities. The how to represent each lexical translation rule with two compact probabilities of the translation rules (e.g. the phrasal translation semantic vectors, and learns how to perform decoding probabilities and the lexical weights in phrase-based using the merging type (swap or monotone) dependent recursive and formal syntax-based models) are all computed based on neural networks that attempt to find the best translation the cooccurrence statistics of the rule's source-and targetsides candidate having the minimal semantic gap with the source in the bilingual corpus.