Hierarchical Concept Discovery Models: A Concept Pyramid Scheme

Deep Learning algorithms have recently gained significant attention due to their impressive performance. However, their high complexity and un-interpretable mode of operation hinders their confident deployment in real-world safety-critical tasks. Our goal is to design a framework that admits a highly interpretable decision making process with respect to human understandable concepts, on multiple levels of granularity. To this end, we propose a novel hierarchical concept discovery formulation leveraging: (i) recent advances in image-text models, and (ii) an innovative formulation for multi-level concept selection via data-driven and sparsity inducing Bayesian arguments. Within this framework, concept information does not solely rely on the similarity between the whole image and general unstructured concepts; instead, we introduce the notion of concept hierarchy to uncover and exploit more granular concept information residing in patch-specific regions of the image scene. As we experimentally show, the proposed construction not only outperforms recent CBM approaches, but also yields a principled framework towards interpetability. The recent advent of multimodal models has greatly popularized the deployment of Deep Learning approaches to a variety of tasks and applications. However, in most cases, deep architectures are treated in an alarming black-box manner: given an input, they produce a particular prediction, with their mode of operation and complexity preventing any potential investigation of their decisionmaking process. This property not only raises serious questions concerning their deployment in safety-critical applications, but at the same time it could actively preclude their adoption in settings that could otherwise benefit societal advances, e.g., medical applications.