Google's AI Overviews Will Always Be Broken. That's How AI Works

A week after its algorithms advised people to eat rocks and put glue on pizza, Google admitted Thursday that it needed to make adjustments to its bold new generative AI search feature. The episode highlights the risks of Google's aggressive drive to commercialize generative AI--and also the treacherous and fundamental limitations of that technology. Google's AI Overviews feature draws on Gemini, a large language model like the one behind OpenAI's ChatGPT, to generate written answers to some search queries by summarizing information found online. The current AI boom is built around LLMs' impressive fluency with text, but the software can also use that facility to put a convincing gloss on untruths or errors. Using the technology to summarize online information promises can make search results easier to digest, but it is hazardous when online sources are contractionary or when people may use the information to make important decisions.