He, Jiaming
Backdoor Attacks against Image-to-Image Networks
Jiang, Wenbo, Li, Hongwei, He, Jiaming, Zhang, Rui, Xu, Guowen, Zhang, Tianwei, Lu, Rongxing
Recently, deep learning-based Image-to-Image (I2I) networks have become the predominant choice for I2I tasks such as image super-resolution and denoising. Despite their remarkable performance, the backdoor vulnerability of I2I networks has not been explored. To fill this research gap, we conduct a comprehensive investigation on the susceptibility of I2I networks to backdoor attacks. Specifically, we propose a novel backdoor attack technique, where the compromised I2I network behaves normally on clean input images, yet outputs a predefined image of the adversary for malicious input images containing the trigger. To achieve this I2I backdoor attack, we propose a targeted universal adversarial perturbation (UAP) generation algorithm for I2I networks, where the generated UAP is used as the backdoor trigger. Additionally, in the backdoor training process that contains the main task and the backdoor task, multi-task learning (MTL) with dynamic weighting methods is employed to accelerate convergence rates. In addition to attacking I2I tasks, we extend our I2I backdoor to attack downstream tasks, including image classification and object detection. Extensive experiments demonstrate the effectiveness of the I2I backdoor on state-of-the-art I2I network architectures, as well as the robustness against different mainstream backdoor defenses.
Talk Too Much: Poisoning Large Language Models under Token Limit
He, Jiaming, Jiang, Wenbo, Hou, Guanyu, Fan, Wenshu, Zhang, Rui, Li, Hongwei
Mainstream poisoning attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of the trigger, we present a poisoning attack against LLMs that is triggered by a generation/output condition-token limitation, which is a commonly adopted strategy by users for reducing costs. The poisoned model performs normally for output without token limitation, while becomes harmful for output with limited tokens. To achieve this objective, we introduce BrieFool, an efficient attack framework. It leverages the characteristics of generation limitation by efficient instruction sampling and poisoning data generation, thereby influencing the behavior of LLMs under target conditions. Our experiments demonstrate that BrieFool is effective across safety domains and knowledge domains. For instance, with only 20 generated poisoning examples against GPT-3.5-turbo, BrieFool achieves a 100% Attack Success Rate (ASR) and a 9.28/10 average Harmfulness Score (HS) under token limitation conditions while maintaining the benign performance.
PLE-SLAM: A Visual-Inertial SLAM Based on Point-Line Features and Efficient IMU Initialization
He, Jiaming, Li, Mingrui, Wang, Yangyang, Wang, Hongyu
Visual-inertial SLAM is crucial in various fields, such as aerial vehicles, industrial robots, and autonomous driving. The fusion of camera and inertial measurement unit (IMU) makes up for the shortcomings of a signal sensor, which significantly improves the accuracy and robustness of localization in challenging environments. This article presents PLE-SLAM, an accurate and real-time visual-inertial SLAM algorithm based on point-line features and efficient IMU initialization. First, we use parallel computing methods to extract features and compute descriptors to ensure real-time performance. Adjacent short line segments are merged into long line segments, and isolated short line segments are directly deleted. Second, a rotation-translation-decoupled initialization method is extended to use both points and lines. Gyroscope bias is optimized by tightly coupling IMU measurements and image observations. Accelerometer bias and gravity direction are solved by an analytical method for efficiency. To improve the system's intelligence in handling complex environments, a scheme of leveraging semantic information and geometric constraints to eliminate dynamic features and A solution for loop detection and closed-loop frame pose estimation using CNN and GNN are integrated into the system. All networks are accelerated to ensure real-time performance. The experiment results on public datasets illustrate that PLE-SLAM is one of the state-of-the-art visual-inertial SLAM systems.
DDN-SLAM: Real-time Dense Dynamic Neural Implicit SLAM with Joint Semantic Encoding
Li, Mingrui, He, Jiaming, Jiang, Guangan, Wang, Hongyu
We propose DDN-SLAM, a real-time dense neural implicit These SLAM systems outperform traditional SLAM methods semantic SLAM system designed for dynamic scenes. in terms of texture details, memory consumption, noise While existing neural implicit SLAM systems perform well handling, and outlier processing. in static scenes, they often encounter challenges in realworld Although current neural implicit SLAM systems have environments with dynamic interferences, leading to achieved good reconstruction results in static scenes [8, ineffective tracking and mapping. DDN-SLAM utilizes the 24, 40, 78], many real-world environments are often affected priors provided by the deep semantic system, combined with by dynamic objects, especially in applications such conditional probability fields, for segmentation.By constructing as robotics or autonomous driving, which involve complex depth-guided static masks and employing joint physical environments and may also have low-texture areas multi-resolution hashing encoding, we ensure fast hole filling or significant changes in lighting and viewing angles. Current and high-quality mapping while mitigating the effects neural implicit SLAM systems are unable to achieve of dynamic information interference. To enhance tracking effective tracking and reliable reconstruction in such environments.