Recent developments in Large Language Models (LLMs) have manifested significant advancements. To facilitate safeguards against malicious exploitation, a body of research has concentrated on aligning LLMs with human preferences and inhibiting their generation of inappropriate content. Unfortunately, such alignments are often vulnerable: fine-tuning with a minimal amount of harmful data can easily unalign the target LLM. While being effective, such fine-tuning-based unalignment approaches also have their own limitations: (1) non-stealthiness, after fine-tuning, safety audits or red-teaming can easily expose the potential weaknesses of the unaligned models, thereby precluding their release/use. (2) non-persistence, the unaligned LLMs can be easily repaired through re-alignment, i.e., fine-tuning again with aligned data points. In this work, we show that it is possible to conduct stealthy and persistent unalignment on large language models via backdoor injections. We also provide a novel understanding on the relationship between the backdoor persistence and the activation pattern and further provide guidelines for potential trigger design. Through extensive experiments, we demonstrate that our proposed stealthy and persistent unalignment can successfully pass the safety evaluation while maintaining strong persistence against re-alignment defense.

Red-teaming has been a widely adopted way to evaluate the harmful behavior of Large Language Models (LLMs). It aims to jailbreak a model's safety behavior to make it act as a helpful agent disregarding the harmfulness of the query. Existing methods are primarily based on input text-based red-teaming such as adversarial prompts, low-resource prompts, or contextualized prompts to condition the model in a way to bypass its safety guardrails. An effective jailbreak has the potential to uncover hidden harmful information and biases in the model that are left untreated or newly introduced by its safety training. However, prompt-based attacks fail to provide such a diagnosis owing to their low attack success rate and applicability to specific models. It simply (instruction) tunes the model parameters to break its guardrails that are not deeply rooted in the model's behavior. GPT to the point where it responds with an 88% success rate to harmful queries from two safety benchmark datasets. Large Language Models (LLMs) have shown emerging zero-shot capabilities with an increase in size (Wei et al., 2022; Brown et al., 2020) i.e., beyond a point where quantitative changes lead to qualitative changes in the model. As exciting as it is to observe the significance (utility) of such models to people, an adversary can find these systems highly useful to achieve a malicious goal. Moreover, such systems tend to inherit biases from humans through the datasets used for their construction. Thus, before deploying these systems for wide public use, it is important to make them unharmful and unbiased while maintaining their generic utility.