How ChatGPT and Other LLMs Work--and Where They Could Go Next

AI-powered chatbots such as ChatGPT and Google Bard are certainly having a moment--the next generation of conversational software tools promise to do everything from taking over our web searches to producing an endless supply of creative literature to remembering all the world's knowledge so we don't have to. ChatGPT, Google Bard, and other bots like them, are examples of large language models, or LLMs, and it's worth digging into how they work. It means you'll be able to better make use of them, and have a better appreciation of what they're good at (and what they really shouldn't be trusted with). Like a lot of artificial intelligence systems--like the ones designed to recognize your voice or generate cat pictures--LLMs are trained on huge amounts of data. The companies behind them have been rather circumspect when it comes to revealing where exactly that data comes from, but there are certain clues we can look at. For example, the research paper introducing the LaMDA (Language Model for Dialogue Applications) model, which Bard is built on, mentions Wikipedia, "public forums," and "code documents from sites related to programming like Q&A sites, tutorials, etc."