Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsal

We initiate the study of Neuro-Symbolic Continual Learning (NeSy-CL), in which the goal is to solve a sequence We introduce Neuro-Symbolic Continual Learning, of neuro-symbolic tasks. As is common in neuro-symbolic where a model has to solve a sequence of (NeSy) prediction (Manhaeve et al., 2018; Xu et al., 2018; neuro-symbolic tasks, that is, it has to map subsymbolic Giunchiglia & Lukasiewicz, 2020; Hoernle et al., 2022; inputs to high-level concepts and compute Ahmed et al., 2022a), the machine is provided prior knowledge predictions by reasoning consistently with relating one or more target labels to symbolic, highlevel prior knowledge. Our key observation is that concepts extracted from sub-symbolic data, and has to neuro-symbolic tasks, although different, often compute a prediction by reasoning over said concepts. The share concepts whose semantics remains stable central challenge of Nesy-CL is that the data distribution over time. Traditional approaches fall short: existing and the knowledge may vary across tasks. E.g., in medical continual strategies ignore knowledge altogether, diagnosis knowledge may encode known relationships between while stock neuro-symbolic architectures possible symptoms and conditions, while different suffer from catastrophic forgetting. We show that tasks are characterized by different distributions of X-ray leveraging prior knowledge by combining neurosymbolic scans, symptoms and conditions. The goal, as in continual architectures with continual strategies learning (CL) (Parisi et al., 2019), is to obtain a model that does help avoid catastrophic forgetting, but also attains high accuracy on new tasks without forgetting what that doing so can yield models affected by reasoning it has already learned under a limited storage budget.