Opening The Black Box--Interpretability In Deep Learning

Editor's Note: See Joris and Matteo at their tutorial "Opening The Black Box -- Interpretability in Deep Learning" at ODSC Europe 2019 this November 20th in London. In the last decade, the application of deep neural networks to long-standing problems has brought a breakthrough in performance and prediction power. However, high accuracy, deriving from the increased model complexity, often comes at the price of loss of interpretability, i.e., many of these models behave as black-boxes and fail to provide explanations on their predictions. While in certain application fields this issue may play a secondary role, in high-risk domains, e.g., health care, it is crucial to build trust in a model and being able to understand its behavior. The definition of the verb interpret is "to explain or tell the meaning of: present in understandable terms" (Merriam- Webster 2019).