RETVec: Resilient and Efficient Text Vectorizer

This paper describes RETV ec, an efficient, resilient, and multilingual text vec-torizer designed for neural-based text processing. RETV ec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETV ec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETV ec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETV ec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks.