Evaluating Morphological Alignment of Tokenizers in 70 Languages

Open in new window