Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations

These "visual" explanations are often insufficient, as the model's actual concept remains elusive. Moreover, without insights into the model's semantic concept, it is difficult --if not impossible-- to intervene on the model's behavior via its explanations, called Explanatory Interactive Learning. Consequently, we propose to intervene on a Neuro-Symbolic scene representation, which allows one to revise the model on the semantic level, e.g. "never focus on the color to make your decision". We compiled a novel confounded visual scene data set, the CLEVR-Hans data set, capturing complex compositions of different objects. The results of our experiments on CLEVR-Hans demonstrate that our semantic explanations, i.e. Figure 1: Neuro-Symbolic explanations are needed to revise compositional explanations at a per-object level, can identify deep learning models from focusing on irrelevant features confounders that are not identifiable using "visual" explanations via global feedback rules.