When Researchers Say Mental Model/Theory of Mind of AI, What Are They Really Talking About?

When researchers claim AI systems possess ToM or mental models, they are fundamentally discussing behavioral predictions and bias corrections rather than genuine mental states. LLMs achieving human-level performance on ToM laboratory tasks, these results are based only on behavioral mimicry. Humans develop theories to explain each other's behaviors (Sellars, 1956). This abstract concept is not related to any specific parameters or rubric; it's the theory of an experienced mental state that we Kosinski (2023) argues that individual artificial neurons in LLMs function like "Chinese rooms", which follow Mechanically speaking, this comparison makes sense. The key is to study how they interact, as humans typically don't question others' cognition during Meanwhile, Strachan et al. (2024) claim that LLM behavior is "indistinguishable from human Researchers like Gu et al. (2024) tested GPT -4 using their SimpleToM dataset.