Goto

Collaborating Authors

 please buy


Ticket-Bench: A Kickoff for Multilingual and Regionalized Agent Evaluation

arXiv.org Artificial Intelligence

Large language models (LLMs) are increasingly deployed as task-oriented agents, where success depends on their ability to generate accurate function calls under realistic, multilingual conditions. However, existing agent evaluations largely overlook cultural and linguistic diversity, often relying on monolingual or na ıvely translated benchmarks. We introduce Ticket-Bench, a benchmark for multilingual agent evaluation in task-oriented scenarios. Ticket-Bench simulates the domain of soccer ticket purchases across six major languages--Portuguese, English, Spanish, German, Italian, and French--using localized teams, cities, and user profiles to provide a higher level of realism. We evaluate a wide range of commercial and open-source LLMs, measuring function-calling accuracy and consistency across languages. Results show that reasoning-oriented models (e.g., GPT -5, Qwen3-235B) dominate performance but still exhibit notable cross-lingual disparities. These findings underscore the need for culturally aware, multilingual benchmarks to guide the development of robust LLM agents.


Someone please buy me this battery-powered, plug-in induction range from CES

Engadget

I don't want or need 99 percent of the stuff on display at CES -- don't get me wrong, most of what we're seeing this week is impressive in one way or another, but nothing had seemed like a necessary addition to my life until I happened upon the Copper Charlie, a battery-equipped induction range that plugs directly into a standard 120V outlet. Yes, among the robots, future cars, bionic exoskeletons and AI everything, the thing that's exited me most is a stove (OK, the fluffball robot is pretty great). Up until a few years ago, I had no idea natural gas cooktops were bad for your health, not to mention the environment. Unfortunately, if your home is set up on gas, it's not easy or cheap to switch up to electric. Nearly all full-sized induction stoves require a dedicated 240V outlet and if you don't have one where the stove goes, a call to a electrician is likely in your future.