We know remarkably little about how AI language models work

A growing number of experts have called for these tests to be ditched, saying they boost AI hype and create "the illusion that [AI language models] have greater capabilities than what truly exists." What stood out to me in Will's story is that we know remarkably little about how AI language models work and why they generate the things they do. With these tests, we're trying to measure and glorify their "intelligence" based on their outputs, without fully understanding how they function under the hood. Our tendency to anthropomorphize makes this messy: "People have been giving human intelligence tests--IQ tests and so on--to machines since the very beginning of AI," says Melanie Mitchell, an artificial-intelligence researcher at the Santa Fe Institute in New Mexico. "The issue throughout has been what it means when you test a machine like this. It doesn't mean the same thing that it means for a human."