Revealed: How Nvidia's 'backseat driver' AI learned to read lips

When Nvidia popped the bonnet on its Co-Pilot "backseat driver" AI at this year's Consumer Electronics Show, most onlookers were struck by its ability to lip-read while tracking CES-going "motorists'" actions within the "car". A slide taken at CES shows the Co-Pilot AI assistant performing four features: facial recognition, head tracking, gaze tracking and lip-reading. The @nvidia AI co-pilot analyzes you through face recognition, head and gaze tracking and lip reading to assist you. The automative AI is part of the GPU-flinger's DRIVE PX 2 platform, which uses sensors and multiple neural networks powered by the grunt of Nvidia's processors. An Nvidia spokesperson has since confirmed in an email to The Register that the lip-reading component was based on research paper [PDF] written by academics from the University of Oxford, Google DeepMind and the Canadian Institute for Advanced Research.