Researchers apply developmental psychology to AI model that predicts object relationships

Humans have no trouble recognizing objects and reasoning about their behaviors -- it's at the core of their cognitive development. Even as children, they group segments into objects based on motion and use concepts of object permanence, solidity, and continuity to explain what has happened and imagine what would happen in other scenarios. Inspired by this, a team of researchers hailing from the MIT-IBM Watson AI Lab, MIT's Computer Science and Artificial Intelligence Laboratory, Alphabet's DeepMind, and Harvard University sought to simplify the problem of visual recognition by introducing a benchmark -- CoLlision Events for Video REpresentation and Reasoning (CLEVRER) -- that draws on inspirations from developmental psychology. CLEVRER contains over 20,000 5-second videos of colliding objects (three shapes of two materials and eight colors) generated by a physics engine and more than 300,000 questions and answers, all focusing on four elements of logical reasoning: descriptive (e.g., "what color"), explanatory ("what's responsible for"), predictive ("what will happen next"), and counterfactual ("what if"). It comes with ground-truth motion traces and event histories for each object in the videos, and with functional programs representing underlying logic that pair with each question.