Machine Translation
A Retrieve-and-Edit Framework for Predicting Structured Outputs
Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren, Percy S. Liang
Neural Information Processing Systems http://nips.cc/
A Tensorized Transformer for Language Modeling
Xindian Ma, Peng Zhang, Shuai Zhang, Nan Duan, Yuexian Hou, Ming Zhou, Dawei Song
Neural Information Processing Systems http://nips.cc/
Language Model Tokenizers Introduce Unfairness Between Languages
Recent language models have shown impressive multilingual performance, even when not explicitly trained for it. Despite this, there are concerns about the quality of their outputs across different languages. In this paper, we show how disparity in the treatment of different languages arises at the tokenization stage, well before a model is even invoked. The same text translated into different languages can have drastically different tok-enization lengths, with differences up to 15 times in some cases. These disparities persist even for tokenizers that are intentionally trained for multilingual support.
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
Liang-Chieh Chen, Maxwell Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, Jon Shlens
Recent progress hasdemonstrated thatsuchmeta-learning methods may exceed scalable human-invented architectures on image classification tasks.
Blockwise Parallel Decoding for Deep Autoregressive Models
Mitchell Stern, Noam Shazeer, Jakob Uszkoreit
To overcome this limitation, we propose a novel blockwise parallel decoding scheme in which we makepredictions for multiple time steps inparallel then back offtothe longest prefix validated byascoring model.
Neural Machine Translation with Soft Prototype
Yiren Wang, Yingce Xia, Fei Tian, Fei Gao, Tao Qin, Cheng Xiang Zhai, Tie-Yan Liu
This work is conducted at Microsoft Research Asia.
On the Dimensionality of Word Embedding
Neural Information Processing Systems http://nips.cc/
Learning Pipelines with Limited Data and Domain Knowledge: A Study in Parsing Physics Problems
Mrinmaya Sachan, Kumar Avinava Dubey, Tom M. Mitchell, Dan Roth, Eric P. Xing
Neural Information Processing Systems http://nips.cc/
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Jiasen Lu, Dhruv Batra, Devi Parikh, Stefan Lee
Neural Information Processing Systems http://nips.cc/
Unified Language Model Pre-training for Natural Language Understanding and Generation
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon
This paper presents anewUNIfied pre-trainedLanguageModel (UNILM)that can be fine-tuned for both natural language understanding and generation tasks.