Theory of Mind Might Have Spontaneously Emerged in Large Language Models

Abstract: We explore the intriguing possibility that theory of mind (ToM), or the uniquely human ability to impute unobservable mental states to others, might have spontaneously emerged in large language models (LLMs). We designed 40 false-belief tasks, considered a gold standard in testing ToM in humans, and administered them to several LLMs. Each task included a falsebelief scenario, three closely matched true-belief controls, and the reversed versions of all four. Smaller and older models solved no tasks; GPT-3-davinci-003 (from November 2022) and ChatGPT-3.5-turbo These findings suggest the intriguing possibility that ToM, previously considered exclusive to humans, may have spontaneously emerged as a byproduct of LLMs' improving language skills. LLMs' Performance dropped but the results and the conclusions remain the same. LLMs' Performance dropped but the results and the conclusions remain the same. Expanded the discussion to present the results in the context of the ongoing debate on whether AI can be credited with human-like mental properties. Code availability and data: The code used to estimate the results, the false-belief tasks, and the instructions given to research assistants can be accessed at https://osf.io/csdhb. Main Text: Many animals excel at using cues such as vocalization, body posture, gaze, or facial expression to predict other animals' behavior and mental states. Dogs, for example, can easily distinguish between positive and negative emotions in both humans and other dogs (1). Yet, humans do not merely respond to observable cues, but also automatically and effortlessly track others' unobservable mental states, such as their knowledge, intentions, beliefs, and desires (2). This ability--typically referred to as "theory of mind" (ToM)--is considered central to human social interactions (3), communication (4), empathy (5), self-consciousness (6), moral judgment (7, 8), and even religious beliefs (9). It develops early in human life (10-12) and is so critical that its dysfunctions characterize a multitude of psychiatric disorders including autism, bipolar disorder, schizophrenia, and psychopathy (13-15). Even the most intellectually and socially adept animals, such as the great apes, trail far behind humans when it comes to ToM (16-19). Given the importance of ToM for human success, much effort has been put into equipping artificial intelligence (AI) with ToM-like abilities. Virtual and physical AI agents would be better and safer if they could impute unobservable mental states to others. The safety of self-driving cars, for example, would greatly increase if they could anticipate the intentions of pedestrians and human drivers. Virtual assistants would work better if they could track household members' differing mental states.