Machine Translation
Generative Neural Machine Translation
Neural Information Processing Systems http://nips.cc/
When does label smoothing help?
Rafael Mรผller, Simon Kornblith, Geoffrey E. Hinton
To explain these observations, we visualize how label smoothing changes therepresentations learned bythepenultimate layerofthenetwork. We show that label smoothing encourages the representations of training examples from thesame class togroup intight clusters. This results inloss ofinformation inthe logits about resemblances between instances ofdifferent classes, which isnecessary for distillation, but does not hurt generalization or calibration of the model'spredictions.
Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Yuan Li, Xiaodan Liang, Zhiting Hu, Eric P. Xing
Neural Information Processing Systems http://nips.cc/
A Retrieve-and-Edit Framework for Predicting Structured Outputs
Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren, Percy S. Liang
Neural Information Processing Systems http://nips.cc/
A Tensorized Transformer for Language Modeling
Xindian Ma, Peng Zhang, Shuai Zhang, Nan Duan, Yuexian Hou, Ming Zhou, Dawei Song
Neural Information Processing Systems http://nips.cc/
Language Model Tokenizers Introduce Unfairness Between Languages
Recent language models have shown impressive multilingual performance, even when not explicitly trained for it. Despite this, there are concerns about the quality of their outputs across different languages. In this paper, we show how disparity in the treatment of different languages arises at the tokenization stage, well before a model is even invoked. The same text translated into different languages can have drastically different tok-enization lengths, with differences up to 15 times in some cases. These disparities persist even for tokenizers that are intentionally trained for multilingual support.
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
Liang-Chieh Chen, Maxwell Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, Jon Shlens
Recent progress hasdemonstrated thatsuchmeta-learning methods may exceed scalable human-invented architectures on image classification tasks.
Blockwise Parallel Decoding for Deep Autoregressive Models
Mitchell Stern, Noam Shazeer, Jakob Uszkoreit
To overcome this limitation, we propose a novel blockwise parallel decoding scheme in which we makepredictions for multiple time steps inparallel then back offtothe longest prefix validated byascoring model.
Neural Machine Translation with Soft Prototype
Yiren Wang, Yingce Xia, Fei Tian, Fei Gao, Tao Qin, Cheng Xiang Zhai, Tie-Yan Liu
This work is conducted at Microsoft Research Asia.
On the Dimensionality of Word Embedding
Neural Information Processing Systems http://nips.cc/