On the Capabilities of Pointer Networks for Deep Deductive Reasoning

The study of architectures and methods for artificial neural networks so that they can learn and perform tasks from the realm of logic-based knowledge representation and reasoning has a long-standing tradition Besold et al. [2017]. This research area is sometimes referred to as "neuro-symbolic integration" (or "neural-symbolic integration") and there are at least two primary rationales that can be found in the literature on the subject. The first is the desire to arrive at systems that combine the robustness and trainability of artificial neural networks with the transparency and interpretability of knowledge-based systems, while at the same time making use of structured background knowledge. The second rationale is more prevalent in cognitive science and lies in addressing the fundamental gap between symbolic and subsymbolic representation and processing, based on the observation that humans perceive much of their own thinking, introspectively, as symbolic, while the physical structure of the brain gives rise to artificial neural networks as a mathematical and computational abstraction. Many of the earlier lines of research on neuro-symbolic integration, discussed primarily from a cognitive science perspective, can be found in Besold et al. [2017]. Of particular interest is the integration of deep learning with logics that are not propositional in nature, since propositional logic is of limited applicability to knowledge representation and reasoning tasks. In the wake of deep learning breakthroughs, fundamental issues around neuro-symbolic integration have recently received increased attention with some progress being made as new approaches emerge. In particular, there has been progress in developing neural networks that can learn to reason.