Google researchers teach AIs to see the important parts of images -- and tell you about them

This week is the Computer Vision and Pattern Recognition conference in Las Vegas, and Google researchers have several accomplishments to present. They've taught computer vision systems to detect the most important person in a scene, pick out and track individual body parts and describe what they see in language that leaves nothing to the imagination. First, let's consider the ability to find "events and key actors" in video -- a collaboration between Google and Stanford. Footage of scenes like basketball games contain dozens or even hundreds of people, but only a few are worth paying attention to. The CV system described in this paper uses a recurrent neural network to create an "attention mask" for every frame, then track relevance of each object as time proceeds. Over time the system is able to pick out not only the most important actor, but potential important actors, and the events with which they are associated.