Words and images

As we rely more on natural language processing to help us navigate our world, it's more important than ever that these artificial intelligence models -- used increasingly in applications such as caption generation for the visually impaired -- remain true to reality. "The issue is that deep learning-based neural language generation models have no guarantees in generating factually correct sentences that are faithful to the input data," said UC Santa Barbara computer scientist William Wang. Over the many iterations it takes for a language generation model to learn how to describe or predict what a scene depicts, elements can creep in, causing phenomena such as errors in data-to-text translations or object hallucinations, in which the caption contains an object or an action that doesn't exist in the image. As a result, unless you have a way of reining in these errors (or you're surrealist painter René Magritte) these mismatches could spell the end of the usefulness of the language generation model being used. "This is a huge problem," said Wang. "Imagine you are using a news summarization system to read earnings reports -- the loss of faithfulness can give you wrong numbers, wrong facts and misinformation. Similarly, if a visually impaired person relies on an image captioning system to see the environment, wrong generation could create serious consequences."