Google turns to machine learning to advance translation of text out in the real world

Google is giving its translation service an upgrade with a new machine learning-powered addition that will allow users to more easily translate text that appears in the real world, like on storefronts, menus, documents, business cards and other items. Instead of covering up the original text with the translation -- the new feature will instead smartly overlay the translated text on top of the image, while also rebuilding the pixels underneath with an AI-generated background to make the process of reading the translation feel more natural. "Often it's that combination of the word plus the context -- like the background image -- that really brings meaning to what you're seeing," explained Cathy Edwards, VP and GM of Google Search, in a briefing ahead of today's announcement. "You don't want to translate a text to cover up that important context that can come through in the images," she said. To make this process work, Google is using a machine learning technology known as generative adversarial networks, otherwise known as GAN models -- the same technology that powers the Magic Eraser feature to remove objects from photos taken on the Google Pixel smartphones.