Tech Companies Are Training AI to Read Your Lips

The task is incredibly challenging--even expert human lip readers are actually pretty poor at word-for-word interpretation. In 2018, Google subsidiary Deepmind published research unveiling its latest full-sentence lip-reading system. The AI achieved a word error rate (the percent of words it got wrong) of 41 percent on videos containing full sentences. Human lip readers viewing a similar sample of video-only clips had word error rates of 93 percent when given no context about the subject matter and 86 percent when given the video's title, subject category, and several words in the sentence. That study was conducted using a large, custom-curated dataset.