Stochastic parrot or world model? How large language models learn

Large language models show impressive capabilities. Are they just superficial statistics – or is there more to them? Systems such as OpenAI's GPT-3 have shown that large language models have capabilities that can make them useful tools in areas as diverse as text processing and programming. With ChatGPT the company has released a model that puts these capabilities in the hands of the general public, creating new challenges for educational institutions, for example. Impressive capabilities quickly lead to the overestimation of AI systems like ChatGPT.