Lexinvariant Language Models Qian Huang

Empirically, we demonstrate that it can indeed attain perplexity comparable to that of a standard language model, given a sufficiently long context.