Overview
Microsoft, Google, xAI give US access to AI models for security testing
Tech giants Microsoft, Google and xAI say they will allow the United States federal government access to their new artificial intelligence models for national security testing. The Center for AI Standards and Innovation (CAISI) at the Department of Commerce announced the agreement on Tuesday amid increasing concerns about the capabilities that Anthropic's newly unveiled Mythos model could give hackers. The agreement fulfils a pledge the administration of US President Donald Trump made in July to partner with technology companies to vet their AI models for "national security risks". Microsoft will work with US government scientists to test AI systems "in ways that probe unexpected behaviors", the company said in a statement. Together they will develop shared data sets and workflows for testing the company's models, the company said.
Extrapolation in Statistical Learning with Extreme Value Theory
Engelke, Sebastian, Gnecco, Nicola, Sabourin, Anne
Extreme value theory provides rigorous theory and statistical tools for extrapolation in machine learning, particularly in settings where traditional methods struggle due to data scarcity in the tails. A broad range of tasks benefit from these advances, including regression and classification beyond the training data, extreme quantile regression, supervised and unsupervised dimension reduction, generative artificial intelligence and anomaly detection. This review synthesizes recent developments in these fields at the intersection of statistical learning and extreme value theory, with a focus on principled methods based on asymptotically motivated representations of the tail of univariate and multivariate distributions. We consider different theoretical frameworks for both asymptotically dependent and independent data and discuss how they translate into efficient statistical methods for extrapolation to extreme regions. By addressing both theoretical and practical aspects, we offer a comprehensive overview of the state-of-the-art in this quickly evolving field, and identify promising directions for future research.
The White House is considering tighter regulation of new AI models
A federal review of new AI models ahead of their public release is being considered as a possible power for that committee, according to the publication's sources. No clear approach has been decided, but the suggested it could mimic what's currently happening within the UK government, where multiple layers of oversight confirm that AI models meet safety standards. There's also still a chance the entire concept fizzles and comes to nothing. If an oversight group is created, it would mark quite a reversal from the hands-off attitude presented in the White House's previously introduced AI Action Plan. The plan appeared willing to offer the AI companies most of the concessions they wanted, although it did leave a lot of potential to create plenty of new problems .
Improving Diffusion-Based Image Synthesis with Context Prediction
Diffusion models are a new class of generative models, and have dramatically promoted image generation with unprecedented quality and diversity. Existing diffusion models mainly try to reconstruct input image from a corrupted one with a pixel-wise or feature-wise constraint along spatial axes. However, such point-based reconstruction may fail to make each predicted pixel/feature fully preserve its neighborhood context, impairing diffusion-based image synthesis. As a powerful source of automatic supervisory signal, context has been well studied for learning representations. Inspired by this, we for the first time propose CONPREDIFF to improve diffusion-based image synthesis with context prediction.
Probabilistic Attention for Interactive Segmentation
We provide a probabilistic interpretation of attention and show that the standard dotproduct attention in transformers is a special case of Maximum APosteriori (MAP) inference. The proposed approach suggests the use of Expectation Maximization algorithms for online adaptation of key and value model parameters. This approach is useful for cases in which external agents, e.g., annotators, provide inference-time information about the correct values of some tokens, e.g., the semantic category of some pixels, and we need for this new information to propagate to other tokens in a principled manner. We illustrate the approach on an interactive semantic segmentation task in which annotators and models collaborate online to improve annotation efficiency. Using standard benchmarks, we observe that key adaptation boosts model performance ( 10% mIoU) in the low feedback regime and value propagation improves model responsiveness in the high feedback regime.
The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only The Falcon LLMTeam
This curation process is believed to be necessary to produce 5 performant models with broad zero-shot generalization abilities. However, as larger 6 models requiring pretraining on trillions of tokens are considered, it is unclear how 7 scalable is curation, and whether we will run out of unique high-quality data soon.