Neural Language Models as Domain-Specific Knowledge Bases

The fundamental challenge of natural language processing (NLP) is resolution of the ambiguity that is present in the meaning of and intent carried by natural language. To resolve ambiguity within a text, algorithms use knowledge from the context within which the text appears. For example, the presence of the sentence "I visited the zoo." before the sentence "I saw a bat" can be used to conclude that bat represents an animal and not a wooden club. While in many situations neighboring text is sufficient for reducing ambiguity, typically it is not sufficient when dealing with text from specialized domains. Processing domain-specific text requires an understanding of a large number of domain-specific concepts and processes that NLP algorithms cannot glean from neighboring text alone.