The Amazing Power of Word Vectors

For today's post, I've drawn material not just from one paper, but from five! The subject matter is'word2vec' – the work of Mikolov et al. at Google on efficient vector representations of words (and what you can do with them). From the first of these papers ('Efficient estimation…') we get a description of theContinuous Bag-of-Words and Continuous Skip-gram models for learning word vectors (we'll talk about what a word vector is in a moment…). From the second paper we get more illustrations of the power of word vectors, some additional information on optimisations for the skip-gram model (hierarchical softmax and negative sampling), and a discussion of applying word vectors to phrases. The third paper ('Linguistic Regularities…') describes vector-oriented reasoning based on word vectors and introduces the famous "King – Man Woman Queen" example.