Evaluating the cognitive capacities of large language models (LLMs) requires overcoming not only anthropomorphic but also anthropocentric biases. This article identifies two types of anthropocentric bias that have been neglected: overlooking how auxiliary factors can impede LLM performance despite competence (Type-I), and dismissing LLM mechanistic strategies that differ from those of humans as not genuinely competent (Type-II). Mitigating these biases necessitates an empirically-driven, iterative approach to mapping cognitive tasks to LLM-specific capacities and mechanisms, which can be done by supplementing carefully designed behavioral experiments with mechanistic studies.

Did you know TNW Conference has a track fully dedicated to bringing the biggest names in tech to showcase inspiring talks from those driving the future of technology this year? Tim Leberecht, who authored this piece, is one of the speakers. Check out the full'Impact' program here. On July 20, 1969, the first human landed on the moon. Fifty years later we are in desperate need for another "moonshot" to tackle some of the pressing and overwhelmingly big issues of our time -- from the climate crisis to the decline of democracy to the upheavals to our labor markets and societies caused by the rise of exponential digital technology -- especially Artificial Intelligence (AI). For the past decade, we put our faith in technology as the ultimate problem-solver, and any kind of innovation was tied to technological advances.