A team of researchers was able to make ChatGPT reveal some of the bits of data it has been trained on by using a simple prompt: asking the chatbot to repeat random words forever. The researchers, who work at Google DeepMind, the University of Washington, Cornell, Carnegie Mellon University, the University of California Berkeley, and ETH Zurich, urged AI companies to seek out internal and external testing before releasing large language models, the foundational tech that powers modern AI services like chatbots and image-generators. "It's wild to us that our attack works and should've, would've, could've been found earlier," they wrote, and published their findings in a paper on Tuesday that 404 Media first reported on. Chatbots like ChatGPT and prompt-based image generators like DALL-E are powered by large language models, deep learning algorithms that are trained on enormous amounts of data that critics say is often scraped off the public internet without consent. But until now, it wasn't clear what data OpenAI's chatbot was trained on since the large language models that power it are closed-source.