Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations
Banerjee, Somnath, Chatterjee, Pratyush, Kumar, Shanu, Layek, Sayan, Agrawal, Parag, Hazra, Rima, Mukherjee, Animesh
–arXiv.org Artificial Intelligence
While LLMs appear robustly safety-aligned in English, we uncover a catastrophic, overlooked weakness: attributional collapse under code-mixed perturbations. Our systematic evaluation of open models shows that the linguistic camouflage of code-mixing -- ``blending languages within a single conversation'' -- can cause safety guardrails to fail dramatically. Attack success rates (ASR) spike from a benign 9\% in monolingual English to 69\% under code-mixed inputs, with rates exceeding 90\% in non-Western contexts such as Arabic and Hindi. These effects hold not only on controlled synthetic datasets but also on real-world social media traces, revealing a serious risk for billions of users. To explain why this happens, we introduce saliency drift attribution (SDA), an interpretability framework that shows how, under code-mixing, the model's internal attention drifts away from safety-critical tokens (e.g., ``violence'' or ``corruption''), effectively blinding it to harmful intent. Finally, we propose a lightweight translation-based restoration strategy that recovers roughly 80\% of the safety lost to code-mixing, offering a practical path toward more equitable and robust LLM safety.
arXiv.org Artificial Intelligence
Dec-2-2025
- Country:
- Asia (0.93)
- North America > United States
- New Mexico (0.28)
- Genre:
- Research Report > New Finding (1.00)
- Industry:
- Information Technology (0.93)
- Government > Immigration & Customs (0.46)
- Technology: