Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language

Neural Information Processing Systems 

However, we still lack an understanding of how a predictive objective shapes such representations. Inspired by recent work in vision neuroscience Hénaff et al. (2019), here we test a