The Impact of Token Granularity on the Predictive Power of Language Model Surprisal

Open in new window