Towards White Box Deep Learning

The main advantages of deep neural networks (DNNs) are their architectural simplicity and automatic feature learning. The latter is crucial for working with unstructured data as developers don't need to design features by hand. However, giving away the control over features leads to black box models - DNNs tend to learn hardly interpretable "shortcut" correlations [17] that leak from train to test [20], hampering alignment and out-of-distribution performance. In particular, this gives rise to adversarial attacks [35] - semantically negligible perturbations of data that arbitrarily change model's predictions. Adversarial vulnerability is a widespread phenomenon (vision [35], segmentation/detection [39], speech recognition [9], tabular data [10], RL [19], NLP [41]) and largely contributes to the general lack of trust in DNNs, substantially limiting their adoption in high-stakes applications such as healthcare, military, autonomous vehicles or cybersecurity. Conversely, the main advantage of hand-designed features is the fine-grained control over model's performance; however, such systems quickly become infeasibly complex. This paper aims to address those issues by reconciling Deep Learning with feature engineering - with the help of locality engineering. Specifically, semantic features are introduced as a general conceptual machinery for controlled dimensionality reduction inside a neural network layer. Figure 1 presents the core idea behind the notion and the rigorous definition is given in Section 4. Implementing a semantic feature predominantly involves encoding appropriate invariants (i.e.