Teaching AI systems to learn language from letters, not words

A new approach to natural language processing (NLP) that teaches neural networks linguistic fundamentals by training them using unsegmented textual input on the interaction between individual letters rather than whole words. Most recurrent neural networks (RNNs) that form the basis of NLP systems are trained on vocabularies of known words. To train RNNs in a way that more closely resembles how humans learn the fundamentals of language, we removed the word boundaries from training data sets and trained the networks at the character (instead of word) level. A multilingual study of this unsupervised character-level language modeling task used data sets of millions of words in English, German, and Italian. It showed that these "near tabula rasa" RNNs develop an impressive spectrum of linguistic knowledge, including segmenting groups of characters into words, distinguishing nouns from verbs, and even inducing simple forms of word meaning.