Modeling Million-byte Sequences with Multiscale Transformers Lili Y u Dániel Simig Colin Flaherty Armen Aghajanyan Luke Zettlemoyer Mike Lewis Meta AI

Oct-9-2025, 12:17:16 GMT–Neural Information Processing Systems 

Sequences of millions of bytes are ubiquitous; for example, music, image, or video files typically consist of multiple megabytes.