Generative AI's Biggest Security Flaw Is Not Easy to Fix

In one experiment in February, security researchers forced Microsoft's Bing chatbot to behave like a scammer. Hidden instructions on a web page the researchers created told the chatbot to ask the person using it to hand over their bank account details. This kind of attack, where concealed information can make the AI system behave in unintended ways, is just the beginning. Hundreds of examples of "indirect prompt injection" attacks have been created since then. This type of attack is now considered one of the most concerning ways that language models could be abused by hackers.