maiden
CMT-Bench: Cricket Multi-Table Generation Benchmark for Probing Robustness in Large Language Models
Upadhyay, Ritam, Ahuja, Naman, Baral, Rishabh, Garimella, Aparna, Gupta, Vivek
LLM Driven text-to-table (T2T) systems often rely on extensive prompt-engineering or iterative event extraction in code-parsable formats, which boosts scores but are computationally expensive and obscure how models actually reason over temporal evolving narratives to summarise key information. We present CMT-Bench, a diagnostic benchmark built from live cricket commentary that requires dynamic table generation across two evolving schemas under a dense, rule-governed policy. CMT-Bench is designed to probe robustness via three semantics-preserving dimensions: (i) extractive-cue ablation to separate extractive shortcuts from state tracking, (ii) temporal prefixing to test long-context stability, and (iii) entity-form perturbations (anonymization, outof-distribution substitutions, role-entangling paraphrases) to assess sensitivity to surface variation. Across diverse long-context stateof-the-art LLMs, we find large drops without extractive summaries, monotonic degradation with input length, and consistent accuracy drop under entity-form changes. Complementary distributional tests confirm significant shifts in numeric error patterns, indicating drift in reasoning rather than mere noise. Our results show that current LLMs are brittle in dynamic Textto-table generation, motivating robustness-first evaluation as a prerequisite for developing efficient and scalable approaches for this task.
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- Europe > Austria > Vienna (0.14)
- North America > United States > Florida > Miami-Dade County > Miami (0.04)
- (7 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Leisure & Entertainment > Sports > Cricket (1.00)
- Leisure & Entertainment > Sports > Soccer (0.93)
Iron Maiden sue video game company for $2m over Ion Maiden game
Iron Maiden are suing video game company 3D Realms over the game Ion Maiden, which they describe as an "incredibly blatant" infringement on their trademark. The lawsuit, which demands $2m (£1.58m) in damages, argues that the game's title will cause "confusion among consumers", "is nearly identical to the Iron Maiden trademark in appearance, sound and overall commercial impression", and "is attempting to trade off on Iron Maiden's notoriety". It argues that the name of the game's protagonist Shelley Harrison is a play on Iron Maiden bassist Steve Harris, and that a skull icon is similar to the band's mascot Eddie. It also argues the game is similar to Iron Maiden's own game Legacy of the Beast and gives examples of fans writing online that they presumed the game was a band tie-in. The game is still in development, and 3D Realms adds that "everyone continues to work diligently on Ion Maiden to deliver the best possible experience later this year".
- Leisure & Entertainment > Games > Computer Games (1.00)
- Law > Intellectual Property & Technology Law (0.70)
- Law > Litigation (0.66)