The way we measure progress in AI is terrible

"It seems to be like the Wild West because we don't really have good evaluation standards," says Anka Reuel, an author of the paper, who is a PhD student in computer science at Stanford University and a member of its Center for AI Safety. A benchmark is essentially a test that an AI takes. It can be in a multiple-choice format like the most popular one, the Massive Multitask Language Understanding benchmark, known as the MMLU, or it could be an evaluation of AI's ability to do a specific task or the quality of its text responses to a set series of questions. AI companies frequently cite benchmarks as testament to a new model's success. "The developers of these models tend to optimize for the specific benchmarks," says Anna Ivanova, professor of psychology at the Georgia Institute of Technology and head of its Language, Intelligence, and Thought (LIT) lab, who was not involved in the Stanford research.