Goto

Collaborating Authors

 Generative AI


Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts

arXiv.org Artificial Intelligence

Large Vision Language Models (VLMs) extend and enhance the perceptual abilities of Large Language Models (LLMs). Despite offering new possibilities for LLM applications, these advancements raise significant security and ethical concerns, particularly regarding the generation of harmful content. While LLMs have undergone extensive security evaluations with the aid of red teaming frameworks, VLMs currently lack a well-developed one. To fill this gap, we introduce Arondight, a standardized red team framework tailored specifically for VLMs. Arondight is dedicated to resolving issues related to the absence of visual modality and inadequate diversity encountered when transitioning existing red teaming methodologies from LLMs to VLMs. Our framework features an automated multi-modal jailbreak attack, wherein visual jailbreak prompts are produced by a red team VLM, and textual prompts are generated by a red team LLM guided by a reinforcement learning agent. To enhance the comprehensiveness of VLM security evaluation, we integrate entropy bonuses and novelty reward metrics. These elements incentivize the RL agent to guide the red team LLM in creating a wider array of diverse and previously unseen test cases. Our evaluation of ten cutting-edge VLMs exposes significant security vulnerabilities, particularly in generating toxic images and aligning multi-modal prompts. In particular, our Arondight achieves an average attack success rate of 84.5\% on GPT-4 in all fourteen prohibited scenarios defined by OpenAI in terms of generating toxic text. For a clearer comparison, we also categorize existing VLMs based on their safety levels and provide corresponding reinforcement recommendations. Our multimodal prompt dataset and red team code will be released after ethics committee approval. CONTENT WARNING: THIS PAPER CONTAINS HARMFUL MODEL RESPONSES.


Scarlett Johansson refused OpenAI job because 'it would be strange' for her kids, 'against my core values'

FOX News

Scarlett Johansson is speaking out about the reasons she turned down the job of voicing OpenAI's chatbot. Last year, OpenAI CEO Sam Altman reached out to the 39-year-old actress about potentially hiring her to voice the ChatGPT 4.0 system. In an interview with The New York Times, Johansson, who voiced the character of Samantha, an artificial intelligence virtual assistant in the 2013 film "Her," recalled that she said, "No, thank you. Not for me," when Altman approached her about the gig. "I felt I did not want to be at the forefront of that," Johansson told the Times.


CVE-LLM : Automatic vulnerability evaluation in medical device industry using large language models

arXiv.org Artificial Intelligence

The healthcare industry is currently experiencing an unprecedented wave of cybersecurity attacks, impacting millions of individuals. With the discovery of thousands of vulnerabilities each month, there is a pressing need to drive the automation of vulnerability assessment processes for medical devices, facilitating rapid mitigation efforts. Generative AI systems have revolutionized various industries, offering unparalleled opportunities for automation and increased efficiency. This paper presents a solution leveraging Large Language Models (LLMs) to learn from historical evaluations of vulnerabilities for the automatic assessment of vulnerabilities in the medical devices industry. This approach is applied within the portfolio of a single manufacturer, taking into account device characteristics, including existing security posture and controls. The primary contributions of this paper are threefold. Firstly, it provides a detailed examination of the best practices for training a vulnerability Language Model (LM) in an industrial context. Secondly, it presents a comprehensive comparison and insightful analysis of the effectiveness of Language Models in vulnerability assessment. Finally, it proposes a new human-in-the-loop framework to expedite vulnerability evaluation processes.


OpenAI's new, lightweight GPT-4o mini model promises an improved ChatGPT experience

Engadget

OpenAI on Thursday released a smaller and more affordable version of its flagship large language model that powers ChatGPT. The new model, called GPT-4o mini, will reportedly cost developers 60 percent less to build AI-powered apps and services with as compared to GPT-3.5 Turbo, Open's smallest model until today. But the big news here is for consumers. GPT-4o mini will replace GPT-3.5 Turbo for free users of ChatGPT starting today -- which means that your baseline ChatGPT experience will improve significantly. OpenAI claimed that GPT-4o mini achieved an 82 percent score on an industry benchmark called the MMLU, which stands for Measuring Massive Multitask Language Understanding, and includes 16,000 multiple-choice questions across 57 academic subjects.


OpenAI Slashes the Cost of Using Its AI With a "Mini" Model

WIRED

OpenAI today announced a cut-price "mini" model that it says will allow more companies and programs to tap into its artificial intelligence. The new model, called GPT-4o mini and available starting today, is 60 percent cheaper than OpenAI's most inexpensive existing model while offering higher performance, the company says. OpenAI characterizes the move as part of an effort to make AI "as broadly accessible as possible," but it also reflects growing competition among AI cloud providers as well as rising interest in small and free open source AI models. Meta is expected to debut the largest version of its very capable free offering, Llama 3, next week. "The whole point of OpenAI is to build and distribute AI safely and make it broadly accessible," Olivier Godement, a product manager at OpenAI responsible for the new model, tells WIRED.


Generative AI Augmented Induction-based Formal Verification

arXiv.org Artificial Intelligence

Generative Artificial Intelligence (GenAI) has demonstrated its capabilities in the present world that reduce human effort significantly. It utilizes deep learning techniques to create original and realistic content in terms of text, images, code, music, and video. Researchers have also shown the capabilities of modern Large Language Models (LLMs) used by GenAI models that can be used to aid hardware development. Formal verification is a mathematical-based proof method used to exhaustively verify the correctness of a design. In this paper, we demonstrate how GenAI can be used in induction-based formal verification to increase the verification throughput.


EnergyDiff: Universal Time-Series Energy Data Generation using Diffusion Models

arXiv.org Artificial Intelligence

High-resolution time series data are crucial for operation and planning in energy systems such as electrical power systems and heating systems. However, due to data collection costs and privacy concerns, such data is often unavailable or insufficient for downstream tasks. Data synthesis is a potential solution for this data scarcity. With the recent development of generative AI, we propose EnergyDiff, a universal data generation framework for energy time series data. EnergyDiff builds on state-of-the-art denoising diffusion probabilistic models, utilizing a proposed denoising network dedicated to high-resolution time series data and introducing a novel Marginal Calibration technique. Our extensive experimental results demonstrate that EnergyDiff achieves significant improvement in capturing temporal dependencies and marginal distributions compared to baselines, particularly at the 1-minute resolution. Additionally, EnergyDiff consistently generates high-quality time series data across diverse energy domains, time resolutions, and at both customer and transformer levels with reduced computational need.


PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformers

arXiv.org Artificial Intelligence

The increased demand for tools that automate the 3D content creation process led to tremendous progress in deep generative models that can generate diverse 3D objects of high fidelity. In this paper, we present PASTA, an autoregressive transformer architecture for generating high quality 3D shapes. PASTA comprises two main components: An autoregressive transformer that generates objects as a sequence of cuboidal primitives and a blending network, implemented with a transformer decoder that composes the sequences of cuboids and synthesizes high quality meshes for each object. Our model is trained in two stages: First we train our autoregressive generative model using only annotated cuboidal parts as supervision and next, we train our blending network using explicit 3D supervision, in the form of watertight meshes. Evaluations on various ShapeNet objects showcase the ability of our model to perform shape generation from diverse inputs \eg from scratch, from a partial object, from text and images, as well size-guided generation, by explicitly conditioning on a bounding box that defines the object's boundaries. Moreover, as our model considers the underlying part-based structure of a 3D object, we are able to select a specific part and produce shapes with meaningful variations of this part. As evidenced by our experiments, our model generates 3D shapes that are both more realistic and diverse than existing part-based and non part-based methods, while at the same time is simpler to implement and train.


Japan news media association demands consent and accuracy from generative AI

The Japan Times

Japan's news industry association issued a statement Wednesday demanding providers of generative artificial intelligence services obtain permits from member media organizations to use their news content and ensure accuracy. The Japan Newspaper Publishers and Editors Association, whose members also include broadcasters, said in the statement that generative AI service providers have expanded their businesses in defiance of the association's repeated requests for them to gain permission. In the RAG services, AI answers in a written form questions asked by users by digging related information out of online sources. Sometimes generated answers are identical with original news stories, or sometimes they are inaccurate due to inappropriate diversion and processing of such original content, the association noted, adding that another problem is that AI does not correct wrong answers. Unless such "freeriding" of content is regulated, media organizations' content will die out, causing irreversible harm to the foundation of democracy and national culture, it warned, urging the government to promptly review laws on intellectual properties.


OpenAI Touts New AI Safety Research. Critics Say It's a Good Step, but Not Enough

WIRED

OpenAI has faced opprobrium in recent months from those who suggest it may be rushing too quickly and recklessly to develop more powerful artificial intelligence. The company appears intent on showing it takes AI safety seriously. Today it showcased research that it says could help researchers scrutinize AI models even as they become more capable and useful. The new technique is one of several ideas related to AI safety that the company has touted in recent weeks. It involves having two AI models engage in a conversation that forces the more powerful one to be more transparent, or "legible," with its reasoning so that humans can understand what it's up to.