South America
Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
Garrod, Connall, Keating, Jonathan P., Thrampoulidis, Christos
Cross-entropy (CE) training loss dominates deep learning practice, yet existing theory often relies on simplifications, either replacing it with squared loss or restricting to convex models, that miss essential behavior. CE and squared loss generate fundamentally different dynamics, and convex linear models cannot capture the complexities of non-convex optimization. We provide an in-depth characterization of multi-class CE optimization dynamics beyond the convex regime by analyzing a canonical two-layer linear neural network with standard-basis vectors as inputs: the simplest non-convex extension for which the implicit bias remained unknown. This model coincides with the unconstrained features model used to study neural collapse, making our work the first to prove that gradient flow on CE converges to the neural collapse geometry. We construct an explicit Lyapunov function that establishes global convergence, despite the presence of spurious critical points in the non-convex landscape. A key insight underlying our analysis is an inconspicuous finding: Hadamard Initialization diagonalizes the softmax operator, freezing the singular vectors of the weight matrices and reducing the dynamics entirely to their singular values. This technique opens a pathway for analyzing CE training dynamics well beyond our specific setting considered here.
Single-Round Scalable Analytic Federated Learning
Bacellar, Alan T. L., Munir, Mustafa, França, Felipe M. G., Lima, Priscila M. V., Marculescu, Radu, John, Lizy K.
Federated Learning (FL) is plagued by two key challenges: high communication overhead and performance collapse on heterogeneous (non-IID) data. Analytic FL (AFL) provides a single-round, data distribution invariant solution, but is limited to linear models. Subsequent non-linear approaches, like DeepAFL, regain accuracy but sacrifice the single-round benefit. In this work, we break this tradeoff. W e propose SAFLe, a framework that achieves scalable non-linear expressivity by introducing a structured head of bucketed features and sparse, grouped embeddings. W e prove this non-linear architecture is mathematically equivalent to a high-dimensional linear regression. This key equivalence allows SAFLe to be solved with AFL's single-shot, invariant aggregation law. Empirically, SAFLe establishes a new state-of-the-art for analytic FL, significantly outperforming both linear AFL and multi-round DeepAFL in accuracy across all benchmarks, demonstrating a highly efficient and scalable solution for federated vision.
How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy
Ponomareva, Natalia, Xu, Zheng, McMahan, H. Brendan, Kairouz, Peter, Rosenblatt, Lucas, Cohen-Addad, Vincent, Guzmán, Cristóbal, McKenna, Ryan, Andrew, Galen, Bie, Alex, Yu, Da, Kurakin, Alex, Zadimoghaddam, Morteza, Vassilvitskii, Sergei, Terzis, Andreas
High quality data is needed to unlock the full potential of AI for end users. However finding new sources of such data is getting harder: most publicly-available human generated data will soon have been used. Additionally, publicly available data often is not representative of users of a particular system -- for example, a research speech dataset of contractors interacting with an AI assistant will likely be more homogeneous, well articulated and self-censored than real world commands that end users will issue. Therefore unlocking high-quality data grounded in real user interactions is of vital interest. However, the direct use of user data comes with significant privacy risks. Differential Privacy (DP) is a well established framework for reasoning about and limiting information leakage, and is a gold standard for protecting user privacy. The focus of this work, \emph{Differentially Private Synthetic data}, refers to synthetic data that preserves the overall trends of source data,, while providing strong privacy guarantees to individuals that contributed to the source dataset. DP synthetic data can unlock the value of datasets that have previously been inaccessible due to privacy concerns and can replace the use of sensitive datasets that previously have only had rudimentary protections like ad-hoc rule-based anonymization. In this paper we explore the full suite of techniques surrounding DP synthetic data, the types of privacy protections they offer and the state-of-the-art for various modalities (image, tabular, text and decentralized). We outline all the components needed in a system that generates DP synthetic data, from sensitive data handling and preparation, to tracking the use and empirical privacy testing. We hope that work will result in increased adoption of DP synthetic data, spur additional research and increase trust in DP synthetic data approaches.
E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testing
Sadhuka, Shuvom, Prinster, Drew, Fannjiang, Clara, Scalia, Gabriele, Regev, Aviv, Wang, Hanchen
Agentic AI systems execute a sequence of actions, such as reasoning steps or tool calls, in response to a user prompt. To evaluate the success of their trajectories, researchers have developed verifiers, such as LLM judges and process-reward models, to score the quality of each action in an agent's trajectory. Although these heuristic scores can be informative, there are no guarantees of correctness when used to decide whether an agent will yield a successful output. Here, we introduce e-valuator, a method to convert any black-box verifier score into a decision rule with provable control of false alarm rates. We frame the problem of distinguishing successful trajectories (that is, a sequence of actions that will lead to a correct response to the user's prompt) and unsuccessful trajectories as a sequential hypothesis testing problem. E-valuator builds on tools from e-processes to develop a sequential hypothesis test that remains statistically valid at every step of an agent's trajectory, enabling online monitoring of agents over arbitrarily long sequences of actions. Empirically, we demonstrate that e-valuator provides greater statistical power and better false alarm rate control than other strategies across six datasets and three agents. We additionally show that e-valuator can be used for to quickly terminate problematic trajectories and save tokens. Together, e-valuator provides a lightweight, model-agnostic framework that converts verifier heuristics into decisions rules with statistical guarantees, enabling the deployment of more reliable agentic systems.
Technical Report on Text Dataset Distillation
Ogawa, Keith Ando, Yamamoto, Bruno Lopes, de Alcantara, Lucas Lauton, Zacarias, Victor, Bollis, Edson, Pellicer, Lucas, Costa, Rosimeire Pereira, Costa, Anna Helena Reali, Jordao, Artur
In the vision domain, dataset distillation arises as a technique to condense a large dataset into a smaller synthetic one that exhibits a similar result in the training process. While image data presents an extensive literature of distillation methods, text dataset distillation has fewer works in comparison. Text dataset distillation initially grew as an adaptation of efforts from the vision universe, as the particularities of the modality became clear obstacles, it rose into a separate branch of research. Several milestones mark the development of this area, such as the introduction of methods that use transformer models, the generation of discrete synthetic text, and the scaling to decoder-only models with over 1B parameters. Despite major advances in modern approaches, the field remains in a maturing phase, with room for improvement on benchmarking standardization, approaches to overcome the discrete nature of text, handling complex tasks, and providing explicit examples of real-world applications. In this report, we review past and recent advances in dataset distillation for text, highlighting different distillation strategies, key contributions, and general challenges.
Modeling Topics and Sociolinguistic Variation in Code-Switched Discourse: Insights from Spanish-English and Spanish-Guaraní
Tyagi, Nemika, Guevara, Nelvin Licona, Kellert, Olga
This study presents an LLM-assisted annotation pipeline for the sociolinguistic and topical analysis of bilingual discourse in two typologically distinct contexts: Spanish-English and Spanish-Guaraní. Using large language models, we automatically labeled topic, genre, and discourse-pragmatic functions across a total of 3,691 code-switched sentences, integrated demographic metadata from the Miami Bilingual Corpus, and enriched the Spanish-Guaraní dataset with new topic annotations. The resulting distributions reveal systematic links between gender, language dominance, and discourse function in the Miami data, and a clear diglossic division between formal Guaraní and informal Spanish in Paraguayan texts. These findings replicate and extend earlier interactional and sociolinguistic observations with corpus-scale quantitative evidence. The study demonstrates that large language models can reliably recover interpretable sociolinguistic patterns traditionally accessible only through manual annotation, advancing computational methods for cross-linguistic and low-resource bilingual research.
Banquet, Royal Family and Starmer on first day of German state visit
The Royal Family hosted the first German state visit to the UK in 27 years - with a state banquet and ceremonial events in Windsor. The Prince and Princess of Wales met Frank-Walter Steinmeier on the tarmac at Heathrow, before King Charles hosted him in a glittering, Christmassy state banquet at Windsor Castle. In a speech delivered in both English and German, the King welcomed the President and his wife, as well as the 150 other guests which included Prime Minister Sir Keir Starmer. In response, President Steinmeier said the King's first visit abroad as monarch to Germany in 2023 was a special symbol of the German-English friendship. The BBC's Russia Editor shares his analysis after five hours of peace talks between the Russians and the US.
Nike, Superdry and Lacoste ads banned over misleading green claims
Adverts for Nike, Superdry and Lacoste have been banned for making misleading claims about their green credentials. The UK's advertising watchdog challenged the brands over the use of the word sustainable in paid-for Google ads which were not backed up by evidence of their sustainability. The Advertising Standards Authority (ASA) identified three adverts from the retailers promising customers sustainable materials, sustainable style and sustainable clothing. The UK's advertising code states that the basis of claims about environmental sustainability must be clear and supported by a high level of substantiation. In each case, it asked the companies for evidence to back up the claims about the sustainability of the products.
Russia-Ukraine war: List of key events, day 1,378
What is in the 28-point US plan for Ukraine? 'Ukraine is running out of men, money and time' Can the US get all sides to end the war? Why is Europe opposing Trump's peace plan? Here's where things stand on Wednesday, December 3: Russian forces attacked Ukraine's Kherson region, using "rocket launchers, mortars and drones", killing a 76-year-old woman and injuring at least two other people, the Kherson Regional Prosecutor's Office said in a post on Telegram. A Russian drone attack killed one person and injured five people in the eastern Ukrainian city of Kramatorsk, the head of the city's military administration, Oleksandr Honcharenko, wrote on Facebook.