Last year in August Google released the Transformer, a novel neural network architecture based on a self-attention mechanism particularly well suited for language understanding. Before the Transformer, most neural network based approaches to machine translation relied on recurrent neural networks (RNNs) which operated sequentially using recurrence. In contrast to RNN-based approaches, the Transformer used no recurrence, instead it processed all words or symbols in the sequence and let each word attend the other word over multiple processing steps using a self-attention mechanism to incorporate context from words farther away. This approach led Transformer to train the recurrent models much faster and yield better translation results than RNNs. "However, on smaller and more structured language understanding tasks, or even simple algorithmic tasks such as copying a string (e.g. to transform an input of "abc" to "abcabc"), the Transformer does not perform very well.", says Stephan Gouws and Mostafa Dehghani from the Google Brain team. Hence this year the team has come up with Universal Transformers, an extension to standard Transformer which is computationally universal using a novel and efficient flavor of parallel-in-time recurrence.

It may not seem obvious, but your translation supply chain represents an enormous data security risk to your business. Particularly when you consider that hundreds of stakeholders could be involved in creating, translating, managing and delivering just one piece of content to customers. It's no surprise then, that security and data privacy have become boardroom issues. But recent advances in machine learning and artificial intelligence (AI) are changing everything, offering a fresh perspective and approach to some of the most difficult security challenges. Here we speak to Matthew Hardy, VP of Customer Solutions at SDL, about the complexities involved with managing a global content supply chain, and how AI brings new and exciting ways for brands to organize content in a way that improves security across all areas of their business.

Grammatical error correction, like other machine learning tasks, greatly benefits from large quantities of high quality training data, which is typically expensive to produce. While writing a program to automatically generate realistic grammatical errors would be difficult, one could learn the distribution of naturallyoccurring errors and attempt to introduce them into other datasets. Initial work on inducing errors in this way using statistical machine translation has shown promise; we investigate cheaply constructing synthetic samples, given a small corpus of human-annotated data, using an off-the-rack attentive sequence-to-sequence model and a straight-forward post-processing procedure. Our approach yields error-filled artificial data that helps a vanilla bi-directional LSTM to outperform the previous state of the art at grammatical error detection, and a previously introduced model to gain further improvements of over 5% $F_{0.5}$ score. When attempting to determine if a given sentence is synthetic, a human annotator at best achieves 39.39 $F_1$ score, indicating that our model generates mostly human-like instances.

Translation apps are getting better, but they're still not perfect, particularly for minority languages. Can artificial intelligence and deep neural networks help iron out the glitches? During the World Cup in Russia this summer there was a dramatic spike in the use of Google Translate, the company says, as fans tried to strike up conversations with their hosts and fellow fans from around the world. The words for "stadium" and "beer" were in particularly high demand. These days the traditional phrasebook is on the way out.

We propose an end-to-end deep learning model for translating free-form natural language instructions to a high-level plan for behavioral robot navigation. The proposed model uses attention mechanisms to connect information from user instructions with a topological representation of the environment. To evaluate this model, we collected a new dataset for the translation problem containing 11,051 pairs of user instructions and navigation plans. Our results show that the proposed model outperforms baseline approaches on the new dataset. Overall, our work suggests that a topological map of the environment can serve as a relevant knowledge base for translating natural language instructions into a sequence of navigation behaviors.

Responses generated by neural conversational models tend to lack informativeness and diversity. We present Adversarial Information Maximization (AIM), an adversarial learning strategy that addresses these two related but distinct problems. To foster response diversity, we leverage adversarial training that allows distributional matching of synthetic and real responses. To improve informativeness, our framework explicitly optimizes a variational lower bound on pairwise mutual information between query and response. Empirical results from automatic and human evaluations demonstrate that our methods significantly boost informativeness and diversity.

Machine translation is not some exotic future technology in its beta testing phase. It is already on your smartphone, laptop and tablet. And it is widely used. Take Google, for instance: it does a billion translations a day for online users. Microsoft introduced automatic, instant translation into Outlook email; Twitter offers translation on most foreign-language tweets.

We introduce a two-player contest for evaluating the safety and robustness of machine learning systems, with a large prize pool. Unlike most prior work in ML robustness, which studies norm-constrained adversaries, we shift our focus to unconstrained adversaries. Defenders submit machine learning models, and try to achieve high accuracy and coverage on non-adversarial data while making no confident mistakes on adversarial inputs. Attackers try to subvert defenses by finding arbitrary unambiguous inputs where the model assigns an incorrect label with high confidence. We propose a simple unambiguous dataset ("bird-or- bicycle") to use as part of this contest. We hope this contest will help to more comprehensively evaluate the worst-case adversarial risk of machine learning models.

Automatic spelling and grammatical correction systems are one of the most widely used tools within natural language applications. In this thesis, we assume the task of error correction as a type of monolingual machine translation where the source sentence is potentially erroneous and the target sentence should be the corrected form of the input. Our main focus in this project is building neural network models for the task of error correction. In particular, we investigate sequence-to-sequence and attention-based models which have recently shown a higher performance than the state-of-the-art of many language processing problems. We demonstrate that neural machine translation models can be successfully applied to the task of error correction. While the experiments of this research are performed on an Arabic corpus, our methods in this thesis can be easily applied to any language.

After half a century of hype and false starts, artificial intelligence may finally be starting to transform the U.S. economy. An example is machine translation, as we found when analyzing eBay's deployment in 2014 of an AI-based tool that learned to translate by digesting millions of lines of eBay data and data from the Web. The aim is to allow eBay sellers and buyers in different countries to more easily connect with one another. The tool detects the location of an eBay user's Internet Protocol address in, say, a Spanish-speaking country and automatically translates the English title of the eBay offering. After eBay unveiled its English-Spanish translator for search queries and item titles, exports on eBay from the United States to Latin America increased by more than 17 percent.