Generative AI
PRIVET: Privacy Metric Based on Extreme Value Theory
Szatkownik, Antoine, Decelle, Aurรฉlien, Seoane, Beatriz, Bereux, Nicolas, Planche, Lรฉo, Charpiat, Guillaume, Yelmen, Burak, Jay, Flora, Furtlehner, Cyril
Deep generative models are often trained on sensitive data, such as genetic sequences, health data, or more broadly, any copyrighted, licensed or protected content. This raises critical concerns around privacy-preserving synthetic data, and more specifically around privacy leakage, an issue closely tied to overfitting. Existing methods almost exclusively rely on global criteria to estimate the risk of privacy failure associated to a model, offering only quantitative non interpretable insights. The absence of rigorous evaluation methods for data privacy at the sample-level may hinder the practical deployment of synthetic data in real-world applications. Using extreme value statistics on nearest-neighbor distances, we propose PRIVET, a generic sample-based, modality-agnostic algorithm that assigns an individual privacy leak score to each synthetic sample. We empirically demonstrate that PRIVET reliably detects instances of memorization and privacy leakage across diverse data modalities, including settings with very high dimensionality, limited sample sizes such as genetic data and even under underfitting regimes. We compare our method to existing approaches under controlled settings and show its advantage in providing both dataset level and sample level assessments through qualitative and quantitative outputs. Additionally, our analysis reveals limitations in existing computer vision embeddings to yield perceptually meaningful distances when identifying near-duplicate samples.
SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability
Xu, Peiyang, Pan, Minzhou, Chen, Zhaorun, Yang, Shuang, Xiao, Chaowei, Li, Bo
With the rapid proliferation of digital media, the need for efficient and transparent safeguards against unsafe content is more critical than ever. Traditional image guardrail models, constrained by predefined categories, often misclassify content due to their pure feature-based learning without semantic reasoning. Moreover, these models struggle to adapt to emerging threats, requiring costly retraining for new threats. To address these limitations, we introduce SafeVision, a novel image guardrail that integrates human-like reasoning to enhance adaptability and transparency. Our approach incorporates an effective data collection and generation framework, a policy-following training pipeline, and a customized loss function. We also propose a diverse QA generation and training strategy to enhance learning effectiveness. SafeVision dynamically aligns with evolving safety policies at inference time, eliminating the need for retraining while ensuring precise risk assessments and explanations. Recognizing the limitations of existing unsafe image benchmarks, which either lack granularity or cover limited risks, we introduce VisionHarm, a high-quality dataset comprising two subsets: VisionHarm Third-party (VisionHarm-T) and VisionHarm Comprehensive(VisionHarm-C), spanning diverse harmful categories. Through extensive experiments, we show that SafeVision achieves state-of-the-art performance on different benchmarks. SafeVision outperforms GPT-4o by 8.6% on VisionHarm-T and by 15.5% on VisionHarm-C, while being over 16x faster. SafeVision sets a comprehensive, policy-following, and explainable image guardrail with dynamic adaptation to emerging threats.
On the Societal Impact of Machine Learning
This PhD thesis investigates the societal impact of machine learning (ML). ML increasingly informs consequential decisions and recommendations, significantly affecting many aspects of our lives. As these data-driven systems are often developed without explicit fairness considerations, they carry the risk of discriminatory effects. The contributions in this thesis enable more appropriate measurement of fairness in ML systems, systematic decomposition of ML systems to anticipate bias dynamics, and effective interventions that reduce algorithmic discrimination while maintaining system utility. I conclude by discussing ongoing challenges and future research directions as ML systems, including generative artificial intelligence, become increasingly integrated into society. This work offers a foundation for ensuring that ML's societal impact aligns with broader social values.
Schrรถdinger bridge for generative AI: Soft-constrained formulation and convergence analysis
Ma, Jin, Tan, Ying, Xu, Renyuan
Generative AI can be framed as the problem of learning a model that maps simple reference measures into complex data distributions, and it has recently found a strong connection to the classical theory of the Schrรถdinger bridge problems (SBPs) due partly to their common nature of interpolating between prescribed marginals via entropy-regularized stochastic dynamics. However, the classical SBP enforces hard terminal constraints, which often leads to instability in practical implementations, especially in high-dimensional or data-scarce regimes. To address this challenge, we follow the idea of the so-called soft-constrained Schrรถdinger bridge problem (SCSBP), in which the terminal constraint is replaced by a general penalty function. This relaxation leads to a more flexible stochastic control formulation of McKean-Vlasov type. We establish the existence of optimal solutions for all penalty levels and prove that, as the penalty grows, both the controls and value functions converge to those of the classical SBP at a linear rate. Our analysis builds on Doob's h-transform representations, the stability results of Schrรถdinger potentials, Gamma-convergence, and a novel fixed-point argument that couples an optimization problem over the space of measures with an auxiliary entropic optimal transport problem. These results not only provide the first quantitative convergence guarantees for soft-constrained bridges but also shed light on how penalty regularization enables robust generative modeling, fine-tuning, and transfer learning.
AI hallucinates because it's trained to fake answers it doesn't know
Earlier today, OpenAI completed a controversial restructuring of its for-profit arm into a public benefit corporation: the latest gust in a whirlwind that has swept up hundreds of billions of dollars of global investment for artificial intelligence (AI) tools. But even as the AI company--founded as a nonprofit, now valued at 500 billion--completes its long-awaited restructuring, a nagging issue with its core offering remains unresolved: hallucinations. Large language models (LLMs) such as those that underpin OpenAI's popular ChatGPT platform are prone to confidently spouting factually incorrect statements. These blips are often attributed to bad input data, but in a preprint posted last month, a team from OpenAI and the Georgia Institute of Technology proves that even with flawless training data, LLMs can never be all-knowing--in part because some questions are just inherently unanswerable. However, that doesn't mean hallucinations are inevitable.
OpenAI Completes Major Reorganization With 135 Billion Microsoft Stake
An illustration photo shows the OpenAI logo displayed on a smartphone with the Microsoft logo in the background in Chongqing, China on Aug. 27, 2025. An illustration photo shows the OpenAI logo displayed on a smartphone with the Microsoft logo in the background in Chongqing, China on Aug. 27, 2025. OpenAI has completed a restructuring, dividing itself into a nonprofit and for-profit entity, the company announced on Tuesday. The nonprofit arm, now called the OpenAI Foundation, will have a $130 billion stake in the for-profit enterprise, a public benefit corporation called OpenAI Group PBC. "The OpenAI Foundation and OpenAI Group will work in concert to advance solutions to hard problems and opportunities posed by AI progress," the company said in its blog post announcing the restructuring. "This includes making intelligence a tool that everyone can benefit from, building safe and aligned systems, turbocharging scientific discovery, and strengthening global cooperation and resilience."
Adobe debuts 'Prompt to Edit' and music tools as its next big AI features
When you purchase through links in our articles, we may earn a small commission. Adobe unveiled new AI additions to its Firefly image generation tool, plus Photoshop, Premiere, and Lightroom. First, there was generative AI, allowing creators, editors and memelords to create artificial worlds with just a few words. Now, Adobe is offering the ability to edit those worlds with Prompt to Edit, a new feature within Firefly, plus audio capabilities. Adobe announced the new capabilities at its MAX conference, where it typically rolls out new capabilities within its Creative Cloud suite as well as Firefly, its AI image generator -- which now includes soundtracks and AI voiceovers.
WhatsApp is banning AI chatbots like ChatGPT soon. Here's why
When you purchase through links in our articles, we may earn a small commission. If you chat with ChatGPT via WhatsApp, you only have a few months left before you'll have to find an alternative. Uh oh, Meta is implementing a change that some users aren't going to like (and others are going to applaud). Due to new guidelines for WhatsApp business accounts, soon AI chatbots will no longer be allowed when they're used as the main purpose of the messenger app. Providers and developers of artificial intelligence or machine learning technologies, including but not limited to large language models, generative artificial intelligence platforms, general-purpose artificial intelligence assistants are strictly prohibited from accessing or using the WhatsApp Business Solution, whether directly or indirectly, for the purposes of providing, delivering, offering, selling, or otherwise making available such technologies when such technologies are the primary (rather than incidental or ancillary) functionality being made available for use, as determined by Meta in its sole discretion.
OpenAI restructures into public-benefit firm, Microsoft takes 27% stake
Microsoft and OpenAI have reached a deal to allow the ChatGPT maker to restructure itself into a public-benefit corporation, valuing OpenAI at $500bn and giving it more freedom in its business operations. The deal, unveiled on Tuesday, removes a major constraint on raising capital for OpenAI that has existed since 2019. As its ChatGPT service exploded in popularity, those limitations had become a notable source of tension between the two companies. Microsoft will still hold a stake of about $135bn, or 27 percent, in OpenAI Group PBC, which will be controlled by the OpenAI Foundation, a nonprofit, the companies said. Microsoft, based in Redmond, Washington in the United States, has invested $13.8bn in OpenAI, with Tuesday's deal implying that the firm had generated a return of nearly 10 times its investment.
OpenAI completes conversion to for-profit business after lengthy legal saga
Sam Altman speaks in San Francisco on 2 June 2025. Sam Altman speaks in San Francisco on 2 June 2025. OpenAI said on Tuesday it had converted its main business into a for-profit corporation, the conclusion of a lengthy and fraught legal saga. A crucial regulator, Kathy Jennings, the Delaware attorney general, said she approved the plan for the startup, which began as a non-profit in 2015, to change to a public benefit corporation, a type of for-profit entity that expresses commitment to bettering society. The company also said it had reorganized its ownership structure and signed a new agreement with its longtime backer Microsoft that gives the software giant a roughly 27% stake in OpenAI's new for-profit corporation, but changes some of the details of their close partnership.