Relationships from Entity Stream

Relational reasoning is a central component of intelligent behavior, but has proven difficult for neural networks to learn. The Relation Network (RN) module was recently proposed by DeepMind to solve such problems, and demo nstrated state-of- the-art results on a number of datasets. However, the RN modu le scales quadrati-cally in the size of the input, since it calculates relations hip factors between every patch in the visual field, including those that do not corresp ond to entities. In this paper, we describe an architecture that enables relati onships to be determined from a stream of entities obtained by an attention mechanism over the input field. The model is trained end-to-end, and demonstrates equivale nt performance with greater interpretability while requiring only a fraction o f the model parameters of the original RN module.