As more companies roll out artificial intelligence (AI) and machine learning (ML) projects, securing them becomes more important. A report released by IBM and Morning Consult in May stated that of more than 7,500 global businesses, 35% of companies are already using AI, up 13% from last year, while another 42% are exploring it. However, almost 20% of companies say that they were having difficulties securing data and that it is slowing down AI adoption. In a survey conducted last spring by Gartner, security concerns were a top obstacle to adopting AI, tied for first place with the complexity of integrating AI solutions into existing infrastructure. According to a paper Microsoft released last spring, 90% of organizations aren't ready to defend themselves against adversarial machine learning.

Today, we are releasing an AI security risk assessment framework as a step to empower organizations to reliably audit, track, and improve the security of the AI systems. In addition, we are providing new updates to Counterfit, our open-source tool to simplify assessing the security posture of AI systems. There is a marked interest in securing AI systems from adversaries. Counterfit has been heavily downloaded and explored by organizations of all sizes--from startups to governments and large-scale organizations--to proactively secure their AI systems. From a different vantage point, the Machine Learning Evasion Competition we organized to help security professionals exercise their muscles to defend and attack AI systems in a realistic setting saw record participation, doubling the amount of participants and techniques than the previous year.

After developing a tool for testing the security of its own AI systems and assessing them for vulnerabilities, Microsoft has decided to open-source it to help organizations verify that that the algorithms they use are "robust, reliable, and trustworthy." Counterfit started as a collection of attack scripts written to target individual AI models, but Microsoft turned it into an automation tool to attack multiple AI systems at scale. Counterfit is also being piloted in the AI development phase to catch vulnerabilities in AI systems before they hit production," Will Pearce and Ram Shankar Siva Kumar from Microsoft's Azure Trustworthy ML team explained. It can be used for penetration testing and red teaming AI systems (by using preloaded published attack algorithms), scanning for vulnerabilities in them, and logging (recording attacks against a target model). After developing a tool for testing the security of its own AI systems and assessing them for vulnerabilities, Microsoft has decided to open-source it to help organizations verify that that the algorithms they use are "robust, reliable, and trustworthy." Counterfit started as a collection of attack scripts written to target individual AI models, but Microsoft turned it into an automation tool to attack multiple AI systems at scale. Counterfit is also being piloted in the AI development phase to catch vulnerabilities in AI systems before they hit production," Will Pearce and Ram Shankar Siva Kumar from Microsoft's Azure Trustworthy ML team explained.

Today, we are releasing Counterfit, an automation tool for security testing AI systems as an open-source project. Counterfit helps organizations conduct AI security risk assessments to ensure that the algorithms used in their businesses are robust, reliable, and trustworthy. AI systems are increasingly used in critical areas such as healthcare, finance, and defense. Consumers must have confidence that the AI systems powering these important domains are secure from adversarial manipulation. For instance, one of the recommendations from Gartner's Top 5 Priorities for Managing AI Risk Within Gartner's MOST Framework published in Jan 20211 is that organizations "Adopt specific AI security measures against adversarial attacks to ensure resistance and resilience," noting that "By 2024, organizations that implement dedicated AI risk management controls will successfully avoid negative AI outcomes twice as often as those that do not."

Artificial intelligence systems take inputs in the form of visuals, audios, texts, etc. As a result, filtering, handling, and detecting malicious inputs and behaviours have become more complicated. Cybersecurity is one of the top priorities of companies worldwide. The increase in the number of AI Security papers from just 617 in 2018 to over 1500 in 2020 (an increase of almost 143% as per an Adversa report) is a testament to the growing importance of cybersecurity. Microsoft has recently announced the release of Counterfit – a tool to test the security of AI systems – as an open-source project.

Microsoft today open-sourced Counterfit, a tool designed to help developers test the security of AI and machine learning systems. The company says that Counterfit can enable organizations to conduct assessments to ensure that the algorithms used in their businesses are robust, reliable, and trustworthy. AI is being increasingly deployed in regulated industries like health care, finance, and defense. But organizations are lagging behind in their adoption of risk mitigation strategies. A Microsoft survey found that 25 out of 28 businesses indicated they don't have the right resources in place to secure their AI systems, and that security professionals are looking for specific guidance in this space.

This document pointed out that the greatest threat to machine learning … permitting organizations to test the security of their artificial intelligence (AI) …