Government
Efficient Detection of Toxic Prompts in Large Language Models
Liu, Yi, Yu, Junzhe, Sun, Huijia, Shi, Ling, Deng, Gelei, Chen, Yuqi, Liu, Yang
Large language models (LLMs) like ChatGPT and Gemini have significantly advanced natural language processing, enabling various applications such as chatbots and automated content generation. However, these models can be exploited by malicious individuals who craft toxic prompts to elicit harmful or unethical responses. These individuals often employ jailbreaking techniques to bypass safety mechanisms, highlighting the need for robust toxic prompt detection methods. Existing detection techniques, both blackbox and whitebox, face challenges related to the diversity of toxic prompts, scalability, and computational efficiency. In response, we propose ToxicDetector, a lightweight greybox method designed to efficiently detect toxic prompts in LLMs. ToxicDetector leverages LLMs to create toxic concept prompts, uses embedding vectors to form feature vectors, and employs a Multi-Layer Perceptron (MLP) classifier for prompt classification. Our evaluation on various versions of the LLama models, Gemma-2, and multiple datasets demonstrates that ToxicDetector achieves a high accuracy of 96.39\% and a low false positive rate of 2.00\%, outperforming state-of-the-art methods. Additionally, ToxicDetector's processing time of 0.0780 seconds per prompt makes it highly suitable for real-time applications. ToxicDetector achieves high accuracy, efficiency, and scalability, making it a practical method for toxic prompt detection in LLMs.
Measure-Theoretic Time-Delay Embedding
Botvinick-Greenhouse, Jonah, Oprea, Maria, Maulik, Romit, Yang, Yunan
The celebrated Takens' embedding theorem provides a theoretical foundation for reconstructing the full state of a dynamical system from partial observations. However, the classical theorem assumes that the underlying system is deterministic and that observations are noise-free, limiting its applicability in real-world scenarios. Motivated by these limitations, we rigorously establish a measure-theoretic generalization that adopts an Eulerian description of the dynamics and recasts the embedding as a pushforward map between probability spaces. Our mathematical results leverage recent advances in optimal transportation theory. Building on our novel measure-theoretic time-delay embedding theory, we have developed a new computational framework that forecasts the full state of a dynamical system from time-lagged partial observations, engineered with better robustness to handle sparse and noisy data. We showcase the efficacy and versatility of our approach through several numerical examples, ranging from the classic Lorenz-63 system to large-scale, real-world applications such as NOAA sea surface temperature forecasting and ERA5 wind field reconstruction.
ODYSSEE: Oyster Detection Yielded by Sensor Systems on Edge Electronics
Lin, Xiaomin, Mange, Vivek, Suresh, Arjun, Neuberger, Bernhard, Palnitkar, Aadi, Campbell, Brendan, Williams, Alan, Baxevani, Kleio, Mallette, Jeremy, Vera, Alhim, Vincze, Markus, Rekleitis, Ioannis, Tanner, Herbert G., Aloimonos, Yiannis
Oysters are a vital keystone species in coastal ecosystems, providing significant economic, environmental, and cultural benefits. As the importance of oysters grows, so does the relevance of autonomous systems for their detection and monitoring. However, current monitoring strategies often rely on destructive methods. While manual identification of oysters from video footage is non-destructive, it is time-consuming, requires expert input, and is further complicated by the challenges of the underwater environment. To address these challenges, we propose a novel pipeline using stable diffusion to augment a collected real dataset with realistic synthetic data. This method enhances the dataset used to train a YOLOv10-based vision model. The model is then deployed and tested on an edge platform in underwater robotics, achieving a state-of-the-art 0.657 mAP@50 for oyster detection on the Aqua2 platform.
Measure Preserving Flows for Ergodic Search in Convoluted Environments
Xu, Albert, Vundurthy, Bhaskar, Gutow, Geordan, Abraham, Ian, Schneider, Jeff, Choset, Howie
Autonomous robotic search has important applications in robotics, such as the search for signs of life after a disaster. When \emph{a priori} information is available, for example in the form of a distribution, a planner can use that distribution to guide the search. Ergodic search is one method that uses the information distribution to generate a trajectory that minimizes the ergodic metric, in that it encourages the robot to spend more time in regions with high information and proportionally less time in the remaining regions. Unfortunately, prior works in ergodic search do not perform well in complex environments with obstacles such as a building's interior or a maze. To address this, our work presents a modified ergodic metric using the Laplace-Beltrami eigenfunctions to capture map geometry and obstacle locations within the ergodic metric. Further, we introduce an approach to generate trajectories that minimize the ergodic metric while guaranteeing obstacle avoidance using measure-preserving vector fields. Finally, we leverage the divergence-free nature of these vector fields to generate collision-free trajectories for multiple agents. We demonstrate our approach via simulations with single and multi-agent systems on maps representing interior hallways and long corridors with non-uniform information distribution. In particular, we illustrate the generation of feasible trajectories in complex environments where prior methods fail.
Towards certifiable AI in aviation: landscape, challenges, and opportunities
Bello, Hymalai, Geiรler, Daniel, Ray, Lala, Mรผller-Divรฉky, Stefan, Mรผller, Peter, Kittrell, Shannon, Liu, Mengxi, Zhou, Bo, Lukowicz, Paul
This fusion can increase efficiency, enhance safety, and improve passenger experience. AI in aviation currently focuses on AI-for-Cabin and non-critical tasks. On the other hand, AI-for-non-Cabin tasks encompass artificial intelligence techniques for the operation of the aircraft, for example, vehicle management or flight control/guidance/management system functions. AI-for-non-Cabin tasks are therefore subject to stringent certification requirements and a thorough and explainable understanding of the target tasks and AI methods to ensure the safety of passengers, flight crew, and aircraft. Moreover, the scope of AI-for-non-Cabin tasks ranges from communication, radar, digital electronics, integrated avionics systems, and navigation, to advanced traffic detection systems, all being considered critical tasks.
Yes, Prime Minister, question order does matter -- and it's certainly not classical! But is it quantum?
In an episode of the satirical British political sitcom Yes, Prime Minister from the 1980s, Sir Humphrey Appleby once explained to Bernard Woolley (two of the characters) how it is possible to get contradictory polling results by asking a series of leading questions beforehand. The polling discussed in the episode concerns whether the public is for or against the reintroduction of national service. Recently, the leading questions outlined by Appleby were put to the public by the market research and polling giant Ipsos, the findings of which have been made public to raise awareness of the fact that people can be misled by means of a such questions [1]. The actual experiment conducted by Ipsos is explained on their web site: "Ipsos interviewed a representative quota sample of 2,158 adults aged 16-75 in Great Britain. Half saw the'Sample A' questions, reflecting a positive view about national service. Half saw'Sample B', reflecting a negative view."
Synthetic Human Memories: AI-Edited Images and Videos Can Implant False Memories and Distort Recollection
Pataranutaporn, Pat, Archiwaranguprok, Chayapatr, Chan, Samantha W. T., Loftus, Elizabeth, Maes, Pattie
AI is increasingly used to enhance images and videos, both intentionally and unintentionally. As AI editing tools become more integrated into smartphones, users can modify or animate photos into realistic videos. This study examines the impact of AI-altered visuals on false memories--recollections of events that didn't occur or deviate from reality. In a pre-registered study, 200 participants were divided into four conditions of 50 each. Participants viewed original images, completed a filler task, then saw stimuli corresponding to their assigned condition: unedited images, AI-edited images, AI-generated videos, or AI-generated videos of AI-edited images. AI-edited visuals significantly increased false recollections, with AI-generated videos of AI-edited images having the strongest effect (2.05x compared to control). Confidence in false memories was also highest for this condition (1.19x compared to control). We discuss potential applications in HCI, such as therapeutic memory reframing, and challenges in ethical, legal, political, and societal domains.
Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Survey
Zhang, Chenyu, Hu, Mingwang, Li, Wenhui, Wang, Lanjun
Recently, the text-to-image diffusion model has gained considerable attention from the community due to its exceptional image generation capability. A representative model, Stable Diffusion, amassed more than 10 million users within just two months of its release. This surge in popularity has facilitated studies on the robustness and safety of the model, leading to the proposal of various adversarial attack methods. Simultaneously, there has been a marked increase in research focused on defense methods to improve the robustness and safety of these models. In this survey, we provide a comprehensive review of the literature on adversarial attacks and defenses targeting text-to-image diffusion models. We begin with an overview of text-to-image diffusion models, followed by an introduction to a taxonomy of adversarial attacks and an in-depth review of existing attack methods. We then present a detailed analysis of current defense methods that improve model robustness and safety. Finally, we discuss ongoing challenges and explore promising future research directions. For a complete list of the adversarial attack and defense methods covered in this survey, please refer to our curated repository at https://github.com/datar001/Awesome-AD-on-T2IDM.
RF Challenge: The Data-Driven Radio Frequency Signal Separation Challenge
Lancho, Alejandro, Weiss, Amir, Lee, Gary C. F., Jayashankar, Tejas, Kurien, Binoy, Polyanskiy, Yury, Wornell, Gregory W.
This paper addresses the critical problem of interference rejection in radio-frequency (RF) signals using a novel, data-driven approach that leverages state-of-the-art AI models. Traditionally, interference rejection algorithms are manually tailored to specific types of interference. This work introduces a more scalable data-driven solution and contains the following contributions. First, we present an insightful signal model that serves as a foundation for developing and analyzing interference rejection algorithms. Second, we introduce the RF Challenge, a publicly available dataset featuring diverse RF signals along with code templates, which facilitates data-driven analysis of RF signal problems. Third, we propose novel AI-based rejection algorithms, specifically architectures like UNet and WaveNet, and evaluate their performance across eight different signal mixture types. These models demonstrate superior performance--exceeding traditional methods like matched filtering and linear minimum mean square error estimation by up to two orders of magnitude in bit-error rate. Fourth, we summarize the results from an open competition hosted at 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024) based on the RF Challenge, highlighting the significant potential for continued advancements in this area. Our findings underscore the promise of deep learning algorithms in mitigating interference, offering a strong foundation for future research.
Promoting Fairness in Link Prediction with Graph Enhancement
Liu, Yezi, Chen, Hanning, Imani, Mohsen
Link prediction is a crucial task in network analysis, but it has been shown to be prone to biased predictions, particularly when links are unfairly predicted between nodes from different sensitive groups. In this paper, we study the fair link prediction problem, which aims to ensure that the predicted link probability is independent of the sensitive attributes of the connected nodes. Existing methods typically incorporate debiasing techniques within graph embeddings to mitigate this issue. However, training on large real-world graphs is already challenging, and adding fairness constraints can further complicate the process. To overcome this challenge, we propose FairLink, a method that learns a fairness-enhanced graph to bypass the need for debiasing during the link predictor's training. FairLink maintains link prediction accuracy by ensuring that the enhanced graph follows a training trajectory similar to that of the original input graph. Meanwhile, it enhances fairness by minimizing the absolute difference in link probabilities between node pairs within the same sensitive group and those between node pairs from different sensitive groups. Our extensive experiments on multiple large-scale graphs demonstrate that FairLink not only promotes fairness but also often achieves link prediction accuracy comparable to baseline methods. Most importantly, the enhanced graph exhibits strong generalizability across different GNN architectures.