Prompt Injection Attacks Are Thwarting AI Hacking Agents

"Context bombing" tricks malicious AI agents into shutting down before they can do harm. Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers' go-to tool for turning AI platforms against their users. A well-phrased command sneaked into an email or calendar invitation is often all it takes to cause the LLM to exfiltrate sensitive data or follow other harmful actions. Now, defenders are embracing the prompt injection, too. Researchers from Tracebit on Monday said they found that placing prompt injections alongside passwords, cryptographic keys, and other secrets stored on Amazon Web Services was often all that was needed to shut down attacks from AI hacking agents.