This Company Uses AI to Outwit Malicious AI

In September 2019, the National Institute of Standards and Technology issued its first-ever warning for an attack on a commercial artificial intelligence algorithm. Security researchers had devised a way to attack a Proofpoint product that uses machine learning to identify spam emails. The system produced email headers that included a "score" of how likely a message was to be spam. But analyzing these scores, along with the contents of messages, made it possible to build a clone of the machine-learning model and craft spam messages that evaded detection. The vulnerability notice may be the first of many.