Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations

Open in new window