Word2Vec -- a baby step in Deep Learning but a giant leap towards Natural Language Processing

Word2Vec model is used for learning vector representations of words called "word embeddings". This is typically done as a preprocessing step, after which the learned vectors are fed into a discriminative model (typically an RNN) to generate predictions and perform all sort of interesting things. Image and audio processing systems work with rich, high-dimensional datasets encoded as vectors of the individual raw pixel-intensities for image data, thus all the information is encoded in the data hence the relation between various entities in the system like (cats and dogs) can be established. But, when it comes to natural language processing systems traditionally it treats words as discrete atomic symbols, and therefore'cat' may be represented as Id537 and'dog' as Id143.These encodings are arbitrary, and provide no useful information to the system regarding the relationships that may exist between the individual symbols. This means that the model can leverage very little of what it has learned about'cats' when it is processing data about'dogs' (such that they are both animals, four-legged, pets, etc.).