Reviews: Learned in Translation: Contextualized Word Vectors

This paper proposes to pretrain sentence encoders for various NLP tasks using machine translation data. In particular, the authors propose to share the whole pretrained sentence encoding model (an LSTM with attention), not just the word embeddings as have been done to great success over the last few years. Evaluations are carried out on sentence classification tasks (sentiment, entailment & question classification) as well as question answering. In all evaluations, the pretrained model outperforms a randomly initialized model with pretrained GloVe embeddings only. This is a good paper that presents a simple idea in a clear, easy to understand manner.