Graph Neural Networks (GNNs) (Kipf & Welling, 2016; Velickovic et al., 2017; Hamilton Graph Neural Networks (GNNs) have achieved et al., 2017), widely recognized as representative methodologies notable success in tasks such as social and transportation in graph-based machine learning, are capable of networks. However, recent studies have deriving high-quality representations from graph data. However, highlighted the vulnerability of GNNs to backdoor despite the remarkable performance of GNNs across attacks, raising significant concerns about various tasks, recent studies (Xi et al., 2021; Zhang et al., their reliability in real-world applications. Despite 2021; Dai et al., 2023; Zhang et al., 2024a) have revealed initial efforts to defend against specific graph that they are vulnerable to backdoor attacks. Backdoor attacks backdoor attacks, existing defense methods face on GNNs typically involve generating and attaching two main challenges: either the inability to establish backdoor triggers to a selected set of target nodes, which are a clear distinction between triggers and subsequently assigned to a specific target class. These triggers, clean nodes, resulting in the removal of many often represented as nodes or subgraphs, can be either clean nodes, or the failure to eliminate the impact predefined or dynamically created using a trigger generator. of triggers, making it challenging to restore the During training on a dataset contaminated with these triggers, target nodes to their pre-attack state. Through empirical due to the graph message-passing paradigm, the GNN analysis of various existing graph backdoor model learns to associate the presence of the trigger with attacks, we observe that the triggers generated by the specific target class. Consequently, during inference, the these methods exhibit over-similarity in both features backdoored model misclassifies test nodes containing the and structure. Based on this observation, we trigger into the target class while maintaining high predictive propose a novel graph backdoor defense method accuracy for clean nodes without triggers.

While previous research backdoored neural networks by changing their parameters, recent work uncovered a more insidious threat: backdoors embedded within the definition of the network's architecture. This involves injecting common architectural components, such as activation functions and pooling layers, to subtly introduce a backdoor behavior that persists even after (full re-)training. However, the full scope and implications of architectural backdoors have remained largely unexplored. Bober-Irizar et al. [2023] introduced the first architectural backdoor; they showed how to create a backdoor for a checkerboard pattern, but never explained how to target an arbitrary trigger pattern of choice. In this work we construct an arbitrary trigger detector which can be used to backdoor an architecture with no human supervision. This leads us to revisit the concept of architecture backdoors and taxonomise them, describing 12 distinct types. To gauge the difficulty of detecting such backdoors, we conducted a user study, revealing that ML developers can only identify suspicious components in common model definitions as backdoors in 37% of cases, while they surprisingly preferred backdoored models in 33% of cases. To contextualize these results, we find that language models outperform humans at the detection of backdoors. Finally, we discuss defenses against architectural backdoors, emphasizing the need for robust and comprehensive strategies to safeguard the integrity of ML systems.

Textual backdoor attack, as a novel attack model, has been shown to be effective in adding a backdoor to the model during training. Defending against such backdoor attacks has become urgent and important. In this paper, we propose AttDef, an efficient attribution-based pipeline to defend against two insertion-based poisoning attacks, BadNL and InSent. Specifically, we regard the tokens with larger attribution scores as potential triggers since larger attribution words contribute more to the false prediction results and therefore are more likely to be poison triggers. Additionally, we further utilize an external pre-trained language model to distinguish whether input is poisoned or not. We show that our proposed method can generalize sufficiently well in two common attack scenarios (poisoning training data and testing data), which consistently improves previous methods. For instance, AttDef can successfully mitigate both attacks with an average accuracy of 79.97% (56.59% up) and 48.34% (3.99% up) under pre-training and post-training attack defense respectively, achieving the new state-of-the-art performance on prediction recovery over four benchmark datasets.

Deep learning models are increasingly used in mobile applications as critical components. Unlike the program bytecode whose vulnerabilities and threats have been widely-discussed, whether and how the deep learning models deployed in the applications can be compromised are not well-understood since neural networks are usually viewed as a black box. In this paper, we introduce a highly practical backdoor attack achieved with a set of reverse-engineering techniques over compiled deep learning models. The core of the attack is a neural conditional branch constructed with a trigger detector and several operators and injected into the victim model as a malicious payload. The attack is effective as the conditional logic can be flexibly customized by the attacker, and scalable as it does not require any prior knowledge from the original model. We evaluated the attack effectiveness using 5 state-of-the-art deep learning models and real-world samples collected from 30 users. The results demonstrated that the injected backdoor can be triggered with a success rate of 93.5%, while only brought less than 2ms latency overhead and no more than 1.4% accuracy decrease. We further conducted an empirical study on real-world mobile deep learning apps collected from Google Play. We found 54 apps that were vulnerable to our attack, including popular and security-critical ones. The results call for the awareness of deep learning application developers and auditors to enhance the protection of deployed models.