Learning Sparse Prototypes for Text Generation

However, there has been a recent move towards non-parametric neural LMs (Guu et al., 2018; Khandelwal et al., 2020b) that generate sentences by first selecting examples from an external datastore. For instance, Khandelwal et al. (2020b) model the token-level probability at test time by interpolating the language model with a kNN distribution from the nearest context-token pairs in the datastore, while Guu et al. (2018) store external memories on sentence level and feature a