The challenges of word embeddings

In recent times deep learning techniques have become more and more prevalent in NLP tasks; just take a look at the list of accepted papers at this year's NAACL conference, and you can't miss it. We've now completely moved away from traditional NLP approaches to focus on deep learning and how it can be leveraged in language problems, as successfully as it has in both image and audio recognition tasks. One of these approaches that has seen great success and is backed by a wave of research papers and funding is the concept of word embeddings. For those of you who aren't familiar with them, word embeddings are essentially dense vector representations of words. Similar to the way a painting might be a representation of a person, a word embedding is a representation of a word, using real-valued numbers.