Knowing Your Nonlinearities: Shapley Interactions Reveal the Underlying Structure of Data

Measuring nonlinear feature interaction is an established This paper investigates Shapley interactions in a number of approach to understanding complex patterns tasks and architectures. We use Shapley interactions as a of attribution in many models. In this paper, case study to illustrate the importance of grounding model we use Shapley Taylor interaction indices interpretations in the underlying structure of the data and the (STII) to analyze the impact of underlying data target models. To this end, we draw connections between structure on model representations in a variety of interaction metrics and various structural properties of the modalities, tasks, and architectures. Considering data in each setting: syntax, tokenization, and idiomatic linguistic structure in masked and auto-regressive expressions in masked and autoregressive language models language models (MLMs and ALMs), we find (MLMs and ALMs, respectively); phoneme articulation differences that STII increases within idiomatic expressions in speech models; and distinctions between edges, and that MLMs scale STII with syntactic distance, foreground, and background pixels in image classifiers. After relying more on syntax in their nonlinear structure introducing our approach to Shapley interactions, we than ALMs do. Our speech model findings apply them in a variety of settings and find the following.