ML Interpretability: Simple Isn't Easy

Machine learning (ML) models, and deep neural networks (DNNs) in particular, are very successful at solving problems both within and outside of science; the latest, spectacular scientific example is the prediction of protein folding (Jumper et al., 2021). However, many of these models are black boxes, and we do not know why they are so successful. As a consequence, the interpretability of ML models - understanding or gaining insight into how they work - is an important area of research in computer science. One kind of effort is towards a better grasp of theoretical properties of ML models, and to formulate what is called a theory of deep learning (Berner et al., 2021; Bahri et al., 2020). Another kind of effort is to provide ML practitioners with tools to understand predictions made by the ML models they deploy. This latter effort often runs under the label of explainable AI (xAI, see, e.g., Adadi and Berrada 2018). Philosophers have also started to pay more attention to interpretability recently; see Beisbart and Räz (2022) for a survey.