Perturbed examples reveal invariances shared by language models

An explosion of work in language is leading to ever-increasing numbers of available natural language processing models, with little understanding of how new models compare to better-understood models. One major reason for this difficulty is saturating benchmark datasets, which may not reflect well differences in model performance in the wild. In this work, we propose a novel framework for comparing two natural language processing models by revealing their shared invariance to interpretable input perturbations that are designed to target a specific linguistic capability (e.g., Synonym-Invariance, Typo-Invariance). Via experiments on models from within the same and across different architecture families, this framework offers a number of insights about how changes in models (e.g., distillation, increase in size, amount of pre-training) affect multiple well-defined linguistic capabilities. Furthermore, we also demonstrate how our framework can enable evaluation of the invariances shared between models that are available as commercial black-box APIs (e.g., InstructGPT family) and models that are relatively better understood (e.g., GPT-2). Across several experiments, we observe that large language models share many of the invariances encoded by models of various sizes, whereas the invariances encoded by large language models are only shared by other large models. Possessing a wide variety of invariances may be a key reason for the recent successes of large language models, and our framework can shed light on the types of invariances that are retained by or emerge in new models. A key reason for the tremendous progress and adoption of natural language processing (NLP) models has been the ready availability of models that can be effectively adapted to diverse downstream tasks and datasets (Wolf et al., 2019). However, with the increasing number of new models, it is difficult to know how new models compare to better-understood ones. This is complicated by the fact that standard benchmark datasets are saturating (Dehghani et al., 2021; Owen, 2023), and small differences on these datasets may in fact correspond to large differences in model performance in the wild (Tay et al., 2022; Zhang et al., 2022; Liu et al., 2023). To enable more comprehensive model comparisons, we propose a novel framework for comparing two natural language processing models by investigating their shared invariance to specific input perturbations. We focus specifically on evaluating invariances that are shared between models, as the invariances encoded by a model implicitly define the features of data that the model deems important and is consequently sensitive to, as well as delineate the features it finds irrelevant.