Language Models for Lexical Inference in Context

Lexical inference (LI) denotes the task of deciding Recently, transfer learning has become ubiquitous whether or not an entailment relation holds between in NLP; Transformer (Vaswani et al., two lexical items. It is therefore related to the detection 2017) language models (LMs) pretrained on large of other lexical relations like hyponymy amounts of textual data (Devlin et al., 2019a; Liu between nouns (Hearst, 1992), e.g., dog animal, et al., 2019) form the basis of a lot of current stateof-the-art or troponymy between verbs (Fellbaum and Miller, models. Besides zero-and few-shot capabilities 1990), e.g., to traipse to walk. Lexical inference (Radford et al., 2019; Brown et al., 2020), in context (LIiC) adds the problem of disambiguating pretrained LMs have also been found to acquire the pair of lexical items in a given context before factual and relational knowledge during pretraining reasoning about the inference question.