Goto

Collaborating Authors

 Media


Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models

arXiv.org Artificial Intelligence

The rapid advancement of Large Vision-Language Models (LVLMs) has enhanced capabilities offering potential applications from content creation to productivity enhancement. Despite their innovative potential, LVLMs exhibit vulnerabilities, especially in generating potentially toxic or unsafe responses. Malicious actors can exploit these vulnerabilities to propagate toxic content in an automated (or semi-) manner, leveraging the susceptibility of LVLMs to deception via strategically crafted prompts without fine-tuning or compute-intensive procedures. Despite the red-teaming efforts and inherent potential risks associated with the LVLMs, exploring vulnerabilities of LVLMs remains nascent and yet to be fully addressed in a systematic manner. This study systematically examines the vulnerabilities of open-source LVLMs, including LLaVA, InstructBLIP, Fuyu, and Qwen, using adversarial prompt strategies that simulate real-world social manipulation tactics informed by social theories. Our findings show that (i) toxicity and insulting are the most prevalent behaviors, with the mean rates of 16.13% and 9.75%, respectively; (ii) Qwen-VL-Chat, LLaVA-v1.6-Vicuna-7b, and InstructBLIP-Vicuna-7b are the most vulnerable models, exhibiting toxic response rates of 21.50%, 18.30% and 17.90%, and insulting responses of 13.40%, 11.70% and 10.10%, respectively; (iii) prompting strategies incorporating dark humor and multimodal toxic prompt completion significantly elevated these vulnerabilities. Despite being fine-tuned for safety, these models still generate content with varying degrees of toxicity when prompted with adversarial inputs, highlighting the urgent need for enhanced safety mechanisms and robust guardrails in LVLM development.


The Theater Stage as Laboratory: Review of Real-Time Comedy LLM Systems for Live Performance

arXiv.org Artificial Intelligence

In this position paper, we review the eclectic recent history of academic and artistic works involving computational systems for humor generation, and focus specifically on live performance. We make the case that AI comedy should be evaluated in live conditions, in front of audiences sharing either physical or online spaces, and under real-time constraints. We further suggest that improvised comedy is therefore the perfect substrate for deploying and assessing computational humor systems. Using examples of successful AI-infused shows, we demonstrate that live performance raises three sets of challenges for computational humor generation: 1) questions around robotic embodiment, anthropomorphism and competition between humans and machines, 2) questions around comedic timing and the nature of audience interaction, and 3) questions about the human interpretation of seemingly absurd AI-generated humor. We argue that these questions impact the choice of methodologies for evaluating computational humor, as any such method needs to work around the constraints of live audiences and performance spaces. These interrogations also highlight different types of collaborative relationship of human comedians towards AI tools.


Jochre 3 and the Yiddish OCR corpus

arXiv.org Artificial Intelligence

We describe the construction of a publicly available Yiddish OCR Corpus, and describe and evaluate the open source OCR tool suite Jochre 3, including an Alto editor for corpus annotation, OCR software for Alto OCR layer generation, and a customizable OCR search engine. The current version of the Yiddish OCR corpus contains 658 pages, 186K tokens and 840K glyphs. The Jochre 3 OCR tool uses various fine-tuned YOLOv8 models for top-down page layout analysis, and a custom CNN network for glyph recognition. It attains a CER of 1.5% on our test corpus, far out-performing all other existing public models for Yiddish. We analyzed the full 660M word Yiddish Book Center with Jochre 3 OCR, and the new OCR is searchable through the Yiddish Book Center OCR search engine.


Gandalf the Red: Adaptive Security for LLMs

arXiv.org Artificial Intelligence

Current evaluations of defenses against prompt attacks in large language model (LLM) applications often overlook two critical factors: the dynamic nature of adversarial behavior and the usability penalties imposed on legitimate users by restrictive defenses. We propose D-SEC (Dynamic Security Utility Threat Model), which explicitly separates attackers from legitimate users, models multi-step interactions, and rigorously expresses the security-utility in an optimizable form. We further address the shortcomings in existing evaluations by introducing Gandalf, a crowd-sourced, gamified red-teaming platform designed to generate realistic, adaptive attack datasets. Using Gandalf, we collect and release a dataset of 279k prompt attacks. Complemented by benign user data, our analysis reveals the interplay between security and utility, showing that defenses integrated in the LLM (e.g., system prompts) can degrade usability even without blocking requests. We demonstrate that restricted application domains, defense-in-depth, and adaptive defenses are effective strategies for building secure and useful LLM applications. Code is available at \href{https://github.com/lakeraai/dsec-gandalf}{\texttt{https://github.com/lakeraai/dsec-gandalf}}.


Logarithmic Memory Networks (LMNs): Efficient Long-Range Sequence Modeling for Resource-Constrained Environments

arXiv.org Artificial Intelligence

Long-range sequence modeling is a crucial aspect of natural language processing and time series analysis. However, traditional models like Recurrent Neural Networks (RNNs) and Transformers suffer from computational and memory inefficiencies, especially when dealing with long sequences. This paper introduces Logarithmic Memory Networks (LMNs), a novel architecture that leverages a hierarchical logarithmic tree structure to efficiently store and retrieve past information. LMNs dynamically summarize historical context, significantly reducing the memory footprint and computational complexity of attention mechanisms from O(n2) to O(log(n)). The model employs a single-vector, targeted attention mechanism to access stored information, and the memory block construction worker (summarizer) layer operates in two modes: a parallel execution mode during training for efficient processing of hierarchical tree structures and a sequential execution mode during inference, which acts as a memory management system. It also implicitly encodes positional information, eliminating the need for explicit positional encodings. These features make LMNs a robust and scalable solution for processing long-range sequences in resource-constrained environments, offering practical improvements in efficiency and scalability. The code is publicly available under the MIT License on GitHub: https://github.com/AhmedBoin/LogarithmicMemory.


Elon Musk, AI and tech titans, venture capitalists invited to pre-inauguration dinner at dawn of Trump era

FOX News

Fox News correspondent William La Jeunesse joins'Fox News Sunday' to discuss the evolution of AI and the push lawmakers are making to regulate it. FIRST ON FOX: A select group of tech industry titans and venture capitalists will gather in Washington, D.C., this week to welcome the incoming Trump administration and celebrate new opportunities for global innovation in artificial intelligence and entrepreneurship. Presidents and CEOs from companies on the cutting edge of AI tech and their big financial backers, along with personnel from the incoming administration, will attend a dinner on Thursday organized by Outside the Box Ventures, a firm founded last year by journalist-turned-investment banker Katherine Tarbox, along with Laurent Bili, the French ambassador to the U.S. The list of those invited to Thursday's dinner includes "DOGE" chief Elon Musk, Silicon Valley investor and GOP mega-donor Peter Thiel, NVCA chief executive Bobby Franklin, incoming White House AI and crypto czar David Sacks, OpenAI's Sam Altman, investor Joe Lonsdale and Narya co-founder Colin Greenspon. "This gathering represents more than discussion. We hope it symbolizes a new chapter in public-private collaboration to harness technology's transformative power for the nation's future," a source close to the planning told Fox News Digital.


The best smart speakers for 2025

Engadget

Smart speakers have become the ultimate multitaskers for your home, combining great sound with the convenience of voice assistants like Alexa, Google Assistant and Siri. Whether you're streaming your favorite playlists, checking the weather, controlling your smart home devices or setting reminders hands free, a good smart speaker can make your day-to-day life a whole lot easier -- and more fun, too. If you're an audiophile, some models prioritize high-quality sound that can fill a room. If you're on a budget, there are plenty of affordable options that still pack in tons of features. And if you're deep into the smart home ecosystem, finding a speaker that seamlessly connects to your devices will be a game-changer. We've picked out the best smart speakers for every need, whether you're after booming bass, a sleek design or advanced voice assistant capabilities.


Stay on top of tech: five ways to take back control, from emails to AI

The Guardian

Asking ChatGPT to write your emails is so two years ago. Generative AI tools are now going beyond the basic text-prompt phase. Take Google's NotebookLM, an experimental "AI research assistant" that lets you upload not just text but also videos, links and PDFs. It will provide a summary of the content, answer questions about it, and even make a podcast-like "AI overview" if you want it to โ€“ all while organising your original sources and notes. As AI tools advance, expect more features like this to be baked into everyday software.


D3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcription

arXiv.org Artificial Intelligence

Diffusion models have been widely used in the generative domain due to their convincing performance in modeling complex data distributions. Moreover, they have shown competitive results on discriminative tasks, such as image segmentation. While diffusion models have also been explored for automatic music transcription, their performance has yet to reach a competitive level. In this paper, we focus on discrete diffusion model's refinement capabilities and present a novel architecture for piano transcription. Our model utilizes Neighborhood Attention layers as the denoising module, gradually predicting the target high-resolution piano roll, conditioned on the finetuned features of a pretrained acoustic model. To further enhance refinement, we devise a novel strategy which applies distinct transition states during training and inference stage of discrete diffusion models. Experiments on the MAESTRO dataset show that our approach outperforms previous diffusion-based piano transcription models and the baseline model in terms of F1 score. Our code is available in https://github.com/hanshounsu/d3rm.


Real-time Verification and Refinement of Language Model Text Generation

arXiv.org Artificial Intelligence

Large language models (LLMs) have shown remarkable performance across a wide range of natural language tasks. However, a critical challenge remains in that they sometimes generate factually incorrect answers. To address this, while many previous work has focused on identifying errors in their generation and further refining them, they are slow in deployment since they are designed to verify the response from LLMs only after their entire generation (from the first to last tokens) is done. Further, we observe that once LLMs generate incorrect tokens early on, there is a higher likelihood that subsequent tokens will also be factually incorrect. To this end, in this work, we propose Streaming-VR (Streaming Verification and Refinement), a novel approach designed to enhance the efficiency of verification and refinement of LLM outputs. Specifically, the proposed Streaming-VR enables on-the-fly verification and correction of tokens as they are being generated, similar to a streaming process, ensuring that each subset of tokens is checked and refined in real-time by another LLM as the LLM constructs its response. Through comprehensive evaluations on multiple datasets, we demonstrate that our approach not only enhances the factual accuracy of LLMs, but also offers a more efficient solution compared to prior refinement methods.