Media
Estonian Native Large Language Model Benchmark
Lillepalu, Helena Grete, Alumäe, Tanel
The availability of LLM benchmarks for the Estonian language is limited, and a comprehensive evaluation comparing the performance of different LLMs on Estonian tasks has yet to be conducted. We introduce a new benchmark for evaluating LLMs in Estonian, based on seven diverse datasets. These datasets assess general and domain-specific knowledge, understanding of Estonian grammar and vocabulary, summarization abilities, contextual comprehension, and more. The datasets are all generated from native Estonian sources without using machine translation. We compare the performance of base models, instruction-tuned open-source models, and commercial models. Our evaluation includes 6 base models and 26 instruction-tuned models. To assess the results, we employ both human evaluation and LLM-as-a-judge methods. Human evaluation scores showed moderate to high correlation with benchmark evaluations, depending on the dataset. Claude 3.7 Sonnet, used as an LLM judge, demonstrated strong alignment with human ratings, indicating that top-performing LLMs can effectively support the evaluation of Estonian-language models.
Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards
Fan, Jiajun, Ren, Roger, Li, Jingyuan, Pandey, Rahul, Shivakumar, Prashanth Gurunath, Bulyko, Ivan, Gandhe, Ankur, Liu, Ge, Gu, Yile
The role of reasoning in Audio Large Language Models remains widely underexplored, as introducing a reasoning process often degrades rather than improves performance during inference, a phenomenon we term test-time inverse scaling, where longer reasoning chains yield progressively worse results. We demonstrate that this stems not from fundamental limitations of reasoning itself, but from inadequate training: models without proper guidance for the reasoning process produce hallucinatory, inconsistent reasoning that accumulates errors over longer chains. To address these challenges, we introduce CESAR (Consistent, Effective, and Scalable Audio Reasoners), shifting from outcome verification to rewarding the reasoning process. Our online reinforcement learning framework employs Group Relative Policy Optimization with a multi-faceted reward suite that incentivizes not only correctness and format but also consistency, structured analytical patterns, causal reasoning, domain-knowledge integration, and calibrated reasoning depth. CESAR resolves test-time inverse scaling, transforming reasoning from detriments into gains while revealing model-specific ``reasoning sweet spots", where performance peaks during test-time scaling. We achieve state-of-the-art results on MMAU Test-mini, substantially outperforming Gemini 2.5 Pro and GPT-4o Audio, and near-human-level performance on MMSU reasoning tasks. Through AI-as-judge evaluations and qualitative comparisons, we provide both quantitative and qualitative validation of our improved reasoning quality. Importantly, enhanced reasoning creates synergistic effects, simultaneously improving multimodal reasoning and perception capabilities. Overall, CESAR establishes a principled method for developing robust and scalable reasoning in Audio LLMs.
What Do AI-Generated Images Want?
W.J.T. Mitchell's influential essay 'What do pictures want?' shifts the theoretical focus away from the interpretative act of understanding pictures and from the motivations of the humans who create them to the possibility that the picture itself is an entity with agency and wants. In this article, I reframe Mitchell's question in light of contemporary AI image generation tools to ask: what do AI-generated images want? Drawing from art historical discourse on the nature of abstraction, I argue that AI-generated images want specificity and concreteness because they are fundamentally abstract. Multimodal text-to-image models, which are the primary subject of this article, are based on the premise that text and image are interchangeable or exchangeable tokens and that there is a commensurability between them, at least as represented mathematically in data. The user pipeline that sees textual input become visual output, however, obscures this representational regress and makes it seem like one form transforms into the other -- as if by magic.
On Optimal Steering to Achieve Exact Fairness
Sharma, Mohit, Deshpande, Amit Jayant, Bhattacharyya, Chiranjib, Shah, Rajiv Ratn
To fix the 'bias in, bias out' problem in fair machine learning, it is important to steer feature distributions of data or internal representations of Large Language Models (LLMs) to ideal ones that guarantee group-fair outcomes. Previous work on fair generative models and representation steering could greatly benefit from provable fairness guarantees on the model output. We define a distribution as ideal if the minimizer of any cost-sensitive risk on it is guaranteed to have exact group-fair outcomes (e.g., demographic parity, equal opportunity)-in other words, it has no fairness-utility trade-off. We formulate an optimization program for optimal steering by finding the nearest ideal distribution in KL-divergence, and provide efficient algorithms for it when the underlying distributions come from well-known parametric families (e.g., normal, log-normal). Empirically, our optimal steering techniques on both synthetic and real-world datasets improve fairness without diminishing utility (and sometimes even improve utility). We demonstrate affine steering of LLM representations to reduce bias in multi-class classification, e.g., occupation prediction from a short biography in Bios dataset (De-Arteaga et al.). Furthermore, we steer internal representations of LLMs towards desired outputs so that it works equally well across different groups.
Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning
Zhang, Haozhen, Feng, Tao, You, Jiaxuan
The rapid emergence of diverse large language models (LLMs) has spurred the development of LLM routers that assign user queries to the most suitable model. However, existing LLM routers typically perform a single-round, one-to-one mapping (\textit{i.e.}, assigning each query to a single model in isolation), which limits their capability to tackle complex tasks that demand the complementary strengths of multiple LLMs. In this paper, we present \textbf{Router-R1}, a reinforcement learning (RL)-based framework that formulates multi-LLM routing and aggregation as a sequential decision process. Router-R1 instantiates the router itself as a capable LLM, leveraging its reasoning ability to interleave "think" actions (internal deliberation) with "route" actions (dynamic model invocation), and integrates each response into its evolving context. To facilitate learning, we employ a lightweight rule-based reward comprising format rewards, final outcome rewards, and a novel cost reward for optimizing the balance between performance and cost, opening a pathway toward enhancing performance-cost trade-offs via RL. Router-R1 also conditions only on simple model descriptors such as pricing, latency, and example performance, enabling strong generalization to unseen model selection. Experiments on seven general and multi-hop QA benchmarks show that Router-R1 outperforms several strong baselines, achieving superior performance while maintaining robust generalization and cost management.
Labor rules out giving tech giants free rein to mine copyright content to train AI
The attorney general, Michelle Rowland, will confirm the decision on Monday, shutting the door on the proposal floated by the Productivity Commission and backed by tech companies. The attorney general, Michelle Rowland, will confirm the decision on Monday, shutting the door on the proposal floated by the Productivity Commission and backed by tech companies. The Albanese government has explicitly ruled out handing tech companies free rein to mine creative content to train their artificial intelligence models, after a fierce backlash from authors and arts and media groups. The attorney general, Michelle Rowland, will confirm the decision on Monday, shutting the door on a contentious proposal floated by the Productivity Commission and backed by tech companies. "Australian creatives are not only world class, but they are also the lifeblood of Australian culture, and we must ensure the right legal protections are in place," Rowland said.
Troublemaking weather pattern is BACK spelling disaster for winter with threat of flooding and wildfires
Two arrested over Louvre'heist of the century' after raid of France's £76million crown jewels Tupac's humiliating intimate disfigurement revealed... and how his lies to cover it up led to his murder Virginia Giuffre's ex boyfriend says she was terrified'shaking with fear' after sex with Prince Andrew fearing'something would happen to her' I've started having heart palpitations. Psychotherapist explains why No Kings rallies consisted of mostly'educated white women' The'marry me' sex move that'll make even the most commitment-phobic of men beg to see you again... and it worked for THREE of my friends Inside Prince Harry and Meghan's final night of freedom at a Halloween party with Princess Eugenie just hours before news of their relationship broke Kristen Bell's friends turn on her with savage disclosures: Insiders reveal poisonous whispers behind her back... as she goes into full diva mode Influencer, 23, speaks out after being arrested for'running interstate drug smuggling network' Kim Kardashian's just been caught in a despicable lie. She can cry all she wants... there's no hiding the truth now: CAROLINE BULLOCK Meghan Markle's fashion faux pas that shocked onlookers during her first overseas tour just months after marrying Prince Harry Inside Andrew's family summit: How Fergie wailed and'melted down' at title loss, Beatrice and Eugenie were'blindsided' and now daughters' assets face'ethics check' to avoid more scandal: BARBARA DAVIES Flamboyant art dealer whose'fake Warhols fooled' Florida's elite is out on bond and cashing in on a flashy new fad Californians being urged to take up arms to deal with'aggressive' invasive species attacking children A disturbing weather pattern could wreak havoc across the US at the end of this year's hurricane season, experts have warned. November tropical storms may be affected by La Nin a, according to Matthew Rosencrans, the lead hurricane seasonal forecaster with the National Oceanic and Atmospheric Administration (NOAA). La Nin a is part of a natural climate cycle known as El Nin o-Southern Oscillation Neutral (ENSO), which alternates between warmer and cooler seawater along the equator in the Pacific Ocean.
Bloody Mary, Bloody Mary, Bloody Mary: How the classic sleepover party game really CAN summon a ghost in your mirror
Tupac's humiliating intimate disfigurement revealed... and how his lies to cover it up led to his murder I've started having heart palpitations. 'Black Ivy League' university looks to expand into crime-riddled Oakland Kristen Bell's friends turn on her with savage disclosures: Insiders reveal poisonous whispers behind her back... as she goes into full diva mode Shooting leaves two dead and 11 injured at large house party with'underage people' in North Carolina Kim Kardashian's just been caught in a despicable lie. She can cry all she wants... there's no hiding the truth now: CAROLINE BULLOCK The'marry me' sex move that'll make even the most commitment-phobic of men beg to see you again... and it worked for THREE of my friends Prosecutor who declined to charge Letitia James with bank fraud fired after'mishandling evidence' Californians being urged to take up arms to deal with'aggressive' invasive species attacking children Inside Andrew's family summit: How Fergie wailed and'melted down' at title loss, Beatrice and Eugenie were'blindsided' and now daughters' assets face'ethics check' to avoid more scandal: BARBARA DAVIES LIZ JONES: I was devastated when my husband cheated. But here's the reason part of me was secretly glad that every woman over-50 will understand Psychotherapist explains why No Kings rallies consisted of mostly'educated white women' Tree optical illusion messes with your mind - you can see the squirrel but can you spot the cat in 30 seconds? Turn off the lights, burn a candle, look into the mirror and say the magic words: 'Bloody Mary, Bloody Mary, Bloody Mary'.