Beyond Text Compression: Evaluating Tokenizers Across Scales

Open in new window