DeepMind & UCL Introduce New Model and Test Set for Inference

A master detective may examine a cigarette discarded in an ashtray and a strand of hair on a lapel and then declare they've solved the murder. Such amazing conclusions are reached through inferential reasoning, a nuanced and uniquely human technique that can form predictions based on connections between seemingly disparate or distanced items and events. Inference is a hot topic for today's neural network researchers, but even SOTA models still struggle to achieve good performance. Now, DeepMind and University College London (UCL) have introduced a new deep network called MEMO which matches SOTA results on Facebook's bAbI dataset for testing text understanding and reasoning, and is the first and only architecture capable of solving long sequence novel reasoning tasks. Due to high similarity between the bAbI training set and test set, neural networks can generate unreliable results through overfitting.