Deep learning has significantly improved state-of-the-art performance for natural language processing tasks like machine translation, summarization, question answering, and text classification. Each of these tasks is typically studied with a specific metric, and performance is often measured on a set of standard benchmark datasets. This has led to the development of architectures designed specifically for those tasks and metrics, but it does not necessarily promote the emergence of general NLP models, those which can perform well across a wide variety of NLP tasks. In order to explore the possibility of such models as well as the tradeoffs that arise in optimizing for them, we introduce the Natural Language Decathlon (decaNLP). The goal of the Decathlon is to explore models that generalize to all ten tasks and investigate how such models differ from those trained for single tasks.

Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten tasks: question answering, machine translation, summarization, natural language inference, sentiment analysis, semantic role labeling, zero-shot relation extraction, goal-oriented dialogue, semantic parsing, and commonsense pronoun resolution. We cast all tasks as question answering over a context. Furthermore, we present a new Multitask Question Answering Network (MQAN) jointly learns all tasks in decaNLP without any task-specific modules or parameters in the multitask setting. MQAN shows improvements in transfer learning for machine translation and named entity recognition, domain adaptation for sentiment analysis and natural language inference, and zero-shot capabilities for text classification. We demonstrate that the MQAN's multi-pointer-generator decoder is key to this success and performance further improves with an anti-curriculum training strategy. Though designed for decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic parsing task in the single-task setting. We also release code for procuring and processing data, training and evaluating models, and reproducing all experiments for decaNLP.

Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typically sees initialization of only the lowest layer of deep models with pretrained word vectors. In this paper, we use a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation (MT) to contextualize word vectors. We show that adding these context vectors (CoVe) improves performance over using only unsupervised word and character vectors on a wide variety of common NLP tasks: sentiment analysis (SST, IMDb), question classification (TREC), entailment (SNLI), and question answering (SQuAD). For fine-grained sentiment analysis and entailment, CoVe improves performance of our baseline models to the state of the art.

This year, we have seen an acceleration of Silicon Valley tech giants opening AI research labs around the world as they seek to gain traction among researchers and fulfill their global ambitions. In the past six months or so, Google brought labs to China and France, Facebook opened labs in Pittsburgh and Seattle, and Microsoft announced plans to open labs near universities in Berkeley, California and Melbourne, Australia. This trend shows no signs of slowing down. Last month, Samsung announced labs in Cambridge, Moscow, and Toronto. This week, Nvidia announced plans to open a new lab in Toronto, while Google shared plans to open a lab in Accra, Ghana, Google's first in Africa and perhaps the first of any tech giant in Africa.

Google Translate has become the internet's go-to resource for short, quick translations from foreign languages. The service was first launched in April 2006, seeing off early competition from the likes of Babel Fish. It now boasts more than 500m users daily worldwide, offering 103 languages. But how exactly does it work? How does Google News actually work?

Sentence simplification aims to improve readability and understandability, based on several operations such as splitting, deletion, and paraphrasing. However, a valid simplified sentence should also be logically entailed by its input sentence. In this work, we first present a strong pointer-copy mechanism based sequence-to-sequence sentence simplification model, and then improve its entailment and paraphrasing capabilities via multi-task learning with related auxiliary tasks of entailment and paraphrase generation. Moreover, we propose a novel 'multi-level' layered soft sharing approach where each auxiliary task shares different (higher versus lower) level layers of the sentence simplification model, depending on the task's semantic versus lexico-syntactic nature. We also introduce a novel multi-armed bandit based training approach that dynamically learns how to effectively switch across tasks during multi-task learning. Experiments on multiple popular datasets demonstrate that our model outperforms competitive simplification systems in SARI and FKGL automatic metrics, and human evaluation. Further, we present several ablation analyses on alternative layer sharing methods, soft versus hard sharing, dynamic multi-armed bandit sampling approaches, and our model's learned entailment and paraphrasing skills.

We present a first attempt to perform attentional word segmentation directly from the speech signal, with the final goal to automatically identify lexical units in a low-resource, unwritten language (UL). Our methodology assumes a pairing between recordings in the UL with translations in a well-resourced language. It uses Acoustic Unit Discovery (AUD) to convert speech into a sequence of pseudo-phones that is segmented using neural soft-alignments produced by a neural machine translation model. Evaluation uses an actual Bantu UL, Mboshi; comparisons to monolingual and bilingual baselines illustrate the potential of attentional word segmentation for language documentation.

Google began their foray into the AI and machine intelligence sectors thanks to the computing power they're able to leverage in the company's data centers. This worked out great as it allowed people all around the world to see how beneficial it was without needing to invest in new hardware. This was when they started to develop their own hardware to handle these computational cycles, and that saved the company a ton of money as this was better designed to handle the tasks required for quick and efficient machine learning algorithms. The latest extension of their progress has come in the form of on-device machine learning hardware. Today, the company showed how Google Translate benefits from using on-device machine learning technology.

Google has rolled out offline downloads for its AI-powered translator. So if you don't have unlimited data or you have a plan that doesn't work internationally, you can now download neural machine translation from Google's Android and iOS apps. But whether you were figuring your way through a foreign menu or deciphering cool storefronts, you may have discovered that the dictionary's quite literal translations don't fully grasp the nuances of a foreign language, even if the language is Spanish or French, which are quite similar to English. The plight doubles when you're trying to decipher a language with a different alphabet and roots, like Russian or Chinese, where even the AI-based translator makes mistakes. Google Translate's offline AI translations will first be available in 59 languages, including English, Arabic, Chinese, German, and Hindi, to name a few.

Disclaimer: Our opinions are our own. Let me quote authors of the framework. In this article we will be using it to train a chatbot. More precisely we will be using the following tutorial for neural machine translation (NMT). If you wonder how an NMT model could be used for a chatbot, please see my previous article ("Own ChatBot Based on Recurrent Neural Network for 6$/6 hours and 100 lines of code.").