Goto

Collaborating Authors

 chameleon




Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models

Neural Information Processing Systems

Large language models (LLMs) have achieved remarkable progress in solving various natural language processing tasks due to emergent reasoning abilities. However, LLMs have inherent limitations as they are incapable of accessing up-to-date information (stored on the Web or in task-specific knowledge bases), using external tools, and performing precise mathematical and logical reasoning.


Chameleon: Adaptive Adversarial Agents for Scaling-Based Visual Prompt Injection in Multimodal AI Systems

Zeeshan, M, Satti, Saud

arXiv.org Artificial Intelligence

Multimodal Artificial Intelligence (AI) systems, particularly Vision-Language Models (VLMs), have become integral to critical applications ranging from autonomous decision-making to automated document processing. As these systems scale, they rely heavily on preprocessing pipelines to handle diverse inputs efficiently. However, this dependency on standard preprocessing operations, specifically image downscaling, creates a significant yet often overlooked security vulnerability. While intended for computational optimization, scaling algorithms can be exploited to conceal malicious visual prompts that are invisible to human observers but become active semantic instructions once processed by the model. Current adversarial strategies remain largely static, failing to account for the dynamic nature of modern agentic workflows. To address this gap, we propose Chameleon, a novel, adaptive adversarial framework designed to expose and exploit scaling vulnerabilities in production VLMs. Unlike traditional static attacks, Chameleon employs an iterative, agent-based optimization mechanism that dynamically refines image perturbations based on the target model's real-time feedback. This allows the framework to craft highly robust adversarial examples that survive standard downscaling operations to hijack downstream execution. We evaluate Chameleon against Gemini 2.5 Flash model. Our experiments demonstrate that Chameleon achieves an Attack Success Rate (ASR) of 84.5% across varying scaling factors, significantly outperforming static baseline attacks which average only 32.1%. Furthermore, we show that these attacks effectively compromise agentic pipelines, reducing decision-making accuracy by over 45% in multi-step tasks. Finally, we discuss the implications of these vulnerabilities and propose multi-scale consistency checks as a necessary defense mechanism.


No lie. The long-nosed Pinocchio chameleon is multiple species.

Popular Science

The long-nosed Pinocchio chameleon is multiple species. Biologists have finally solved the century-old reptilian mystery. Breakthroughs, discoveries, and DIY tips sent every weekday. For nearly 150 years, zoologists have taken the Pinocchio chameleon () at face value.. However, a recent reexamination detailed in reveals that the chameleon is actually multiple species with elongated snouts worthy of the nickname.


Watermarking Autoregressive Image Generation

Jovanović, Nikola, Labiad, Ismail, Souček, Tomáš, Vechev, Martin, Fernandez, Pierre

arXiv.org Artificial Intelligence

Watermarking the outputs of generative models has emerged as a promising approach for tracking their provenance. Despite significant interest in autoregressive image generation models and their potential for misuse, no prior work has attempted to watermark their outputs at the token level. In this work, we present the first such approach by adapting language model watermarking techniques to this setting. We identify a key challenge: the lack of reverse cycle-consistency (RCC), wherein re-tokenizing generated image tokens significantly alters the token sequence, effectively erasing the watermark. To address this and to make our method robust to common image transformations, neural compression, and removal attacks, we introduce (i) a custom tokenizer-detokenizer finetuning procedure that improves RCC, and (ii) a complementary watermark synchronization layer. As our experiments demonstrate, our approach enables reliable and robust watermark detection with theoretically grounded p-values. Code and models are available at https://github.com/facebookresearch/wmar.


HeroFilter: Adaptive Spectral Graph Filter for Varying Heterophilic Relations

Zhang, Shuaicheng, Wang, Haohui, Lin, Junhong, Guo, Xiaojie, Zhu, Yada, Zhang, Si, Fu, Dongqi, Zhou, Dawei

arXiv.org Artificial Intelligence

Graph heterophily, where connected nodes have different labels, has attracted significant interest recently. Most existing works adopt a simplified approach - using low-pass filters for homophilic graphs and high-pass filters for heterophilic graphs. However, we discover that the relationship between graph heterophily and spectral filters is more complex - the optimal filter response varies across frequency components and does not follow a strict monotonic correlation with heterophily degree. This finding challenges conventional fixed filter designs and suggests the need for adaptive filtering to preserve expressiveness in graph embeddings. Formally, natural questions arise: Given a heterophilic graph G, how and to what extent will the varying heterophily degree of G affect the performance of GNNs? How can we design adaptive filters to fit those varying heterophilic connections? Our theoretical analysis reveals that the average frequency response of GNNs and graph heterophily degree do not follow a strict monotonic correlation, necessitating adaptive graph filters to guarantee good generalization performance. Hence, we propose [METHOD NAME], a simple yet powerful GNN, which extracts information across the heterophily spectrum and combines salient representations through adaptive mixing. [METHOD NAME]'s superior performance achieves up to 9.2% accuracy improvement over leading baselines across homophilic and heterophilic graphs.



Automating Steering for Safe Multimodal Large Language Models

Wu, Lyucheng, Wang, Mengru, Xu, Ziwen, Cao, Tri, Oo, Nay, Hooi, Bryan, Deng, Shumin

arXiv.org Artificial Intelligence

Recent progress in Multimodal Large Language Models (MLLMs) has unlocked powerful cross-modal reasoning abilities, but also raised new safety concerns, particularly when faced with adversarial multimodal inputs. To improve the safety of MLLMs during inference, we introduce a modular and adaptive inference-time intervention technology, AutoSteer, without requiring any fine-tuning of the underlying model. AutoSteer incorporates three core components: (1) a novel Safety Awareness Score (SAS) that automatically identifies the most safety-relevant distinctions among the model's internal layers; (2) an adaptive safety prober trained to estimate the likelihood of toxic outputs from intermediate representations; and (3) a lightweight Refusal Head that selectively intervenes to modulate generation when safety risks are detected. Experiments on LLaVA-OV and Chameleon across diverse safety-critical benchmarks demonstrate that AutoSteer significantly reduces the Attack Success Rate (ASR) for textual, visual, and cross-modal threats, while maintaining general abilities. These findings position AutoSteer as a practical, interpretable, and effective framework for safer deployment of multimodal AI systems.


A chameleon's 'ballistic tongue' may inspire blood clot-clearing robots

Popular Science

Environment Animals Wildlife A chameleon's'ballistic tongue' may inspire blood clot-clearing robots Chameleons and salamanders can fire their tongues as fast as 16 feet/second. Breakthroughs, discoveries, and DIY tips sent every weekday. The sticky, slimy tongues of chameleons and salamanders may not sound like a great inspiration for engineering projects or medical innovations. But according to researchers at the University of South Florida, the same biological mechanics used to capture and devour bugs could accomplish similar feats inside your bloodstream--and even in outer space. Chameleons prefer to stick to warmer climates amid branchy trees and bushes, while salamanders mostly keep to moist, shaded environments such as decaying leaf debris and dark caves.