Recurrent Relational Networks

Neural Information Processing Systems

This paper is concerned with learning to solve tasks that require a chain of interde- pendent steps of relational inference, like answering complex questions about the relationships between objects, or solving puzzles where the smaller elements of a solution mutually constrain each other. We introduce the recurrent relational net- work, a general purpose module that operates on a graph representation of objects. As a generalization of Santoro et al. [2017]'s relational network, it can augment any neural network model with the capacity to do many-step relational reasoning. We achieve state of the art results on the bAbI textual question-answering dataset with the recurrent relational network, consistently solving 20/20 tasks. As bAbI is not particularly challenging from a relational reasoning point of view, we introduce Pretty-CLEVR, a new diagnostic dataset for relational reasoning.


Relational recurrent neural networks

Neural Information Processing Systems

Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods. It is unclear, however, whether they also have an ability to perform complex relational reasoning with the information they remember. Here, we first confirm our intuitions that standard memory architectures may struggle at tasks that heavily involve an understanding of the ways in which entities are connected -- i.e., tasks involving relational reasoning. We then improve upon these deficits by using a new memory module -- a Relational Memory Core (RMC) -- which employs multi-head dot product attention to allow memories to interact. Finally, we test the RMC on a suite of tasks that may profit from more capable relational reasoning across sequential information, and show large gains in RL domains (BoxWorld & Mini PacMan), program evaluation, and language modeling, achieving state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord datasets.


DeepMind takes a shot at teaching AI to reason with relational networks

#artificialintelligence

Analysis The ability to think logically and to reason is key to intelligence. When this can be replicated in machines, it will no doubt make AI smarter. But it's a difficult problem, and current methods used in deep learning aren't advanced enough. Deep learning is good for processing information, but it can struggle with reasoning. Enter a different player to the game: relational networks, or RNs.


DeepMind takes a shot at teaching AI to reason with relational networks

#artificialintelligence

The latest paper by DeepMind, Alphabet's British AI outfit, attempts to enable machines to reason by tacking on RNs to convolutional neural networks and recurrent neural networks, both traditionally used for computer vision and natural language processing. "It is not that deep learning is unsuited to reasoning tasks – more that the correct deep learning architectures, or modules, did not exist to enable general relational reasoning. "State-of-the-art results on CLEVR using standard visual question answering architectures are 68.5 per cent, compared to 92.5 per cent for humans. The RN managed to pass 18 out of 20 tasks, beating previous attempts that used memory networks used by Facebook and DeepMind's differentiable neural computer.


Recurrent Relational Networks

Neural Information Processing Systems

This paper is concerned with learning to solve tasks that require a chain of interde- pendent steps of relational inference, like answering complex questions about the relationships between objects, or solving puzzles where the smaller elements of a solution mutually constrain each other. We introduce the recurrent relational net- work, a general purpose module that operates on a graph representation of objects. As a generalization of Santoro et al. [2017]’s relational network, it can augment any neural network model with the capacity to do many-step relational reasoning. We achieve state of the art results on the bAbI textual question-answering dataset with the recurrent relational network, consistently solving 20/20 tasks. As bAbI is not particularly challenging from a relational reasoning point of view, we introduce Pretty-CLEVR, a new diagnostic dataset for relational reasoning. In the Pretty- CLEVR set-up, we can vary the question to control for the number of relational reasoning steps that are required to obtain the answer. Using Pretty-CLEVR, we probe the limitations of multi-layer perceptrons, relational and recurrent relational networks. Finally, we show how recurrent relational networks can learn to solve Sudoku puzzles from supervised training data, a challenging task requiring upwards of 64 steps of relational reasoning. We achieve state-of-the-art results amongst comparable methods by solving 96.6% of the hardest Sudoku puzzles.