Machine learning, a subset of artificial intelligence, is the practice of using algorithms and large data sets or Big Data to develop insights ranging from which movie a Netflix user may want to watch next to recommendations about cybersecurity incident handling. According to consulting firm McKinsey, "the unmanageable volume and complexity of the big data that the world is now swimming in have increased the potential of machine learning--and the need for it." For security professionals, machine learning capabilities can increase responder productivity and enable leaner, more efficient security operations. Humans however, not machines, must direct and guide machine learning algorithms to achieve the business goals and objectives that the computers are given. The best way to understand how machine learning can be beneficial for security analysts is to perhaps look at another field with similar operational efficiency goals that is currently taking advantage of Big Data, and prospering - Marketing.
The things that make big data what it is – high velocity, variety, and volume – make it a challenge to defend. And it presents a tempting target for potential attackers. But big data technologies are also being used to help cybersecurity, since many of the same tools and approaches can be used to collect log and incident data, process it quickly, and spot suspicious activity. "Modern cybersecurity solutions are mostly driven by big data," said Bogdan Botezatu, senior threat analyst at Bitdefender. To start with, all the major anti-virus and endpoint protection vendors, as well as network security and firewall providers, train their systems on the massive volumes of malware and known attack paths that they have collected.
Palo Alto Network is spending $560 million for privately held Demisto in a deal designed to build out the company's application framework strategy and consolidate the security tools within an enterprise. Demisto focuses on security orchestration, automation and response (SOAR) tools. SOAR is a hot market in security. The plan for Palo Alto is to take Demisto's technology and combine it with its application framework. Palo Alto Networks application framework is an ecosystem that allows third parties to build security apps on its data and security tools.
Threat actors are increasingly adopting security automation and machine learning – security teams will have to follow suit, or risk falling behind. Many organizations still conduct incident response based on manual processes. Many playbooks that we have seen in our customer base, for example, hand off to other stakeholders within the organization to wait for additional forensic data, and to execute remediation and containment actions. While this may seem like good practice to avoid inadvertent negative consequences such as accidentally shutting down critical systems or locking out innocent users, it also means that many attacks are not contained in a sufficiently short time to avoid the worst of their consequences. Reports are mounting about threat actors and hackers leveraging security automation and machine learning to increase the scale and volume, as well as the velocity of attacks.
In this age of advanced persistent threats, waiting for traditional threat management solutions like IDS and SIEM to flag incidents and threats is simply not enough anymore. "We live in a world where the adversaries will persist in getting into an organizations environment, and they only have to be successful once. And, on average, companies are breached for more than 200 days before they realize they are compromised," notes Mark Terenzoni, CEO at Sqrrl, a company dedicated to simplifying effective cyber threat hunting. Its threat hunting platform, Sqrrl Enterprise, has made the company a prominent name in the cybersecurity field. Threat hunting is steadily receiving more and more attention now that everyone wants to move away from a reactive posture and towards a proactive hunting methodology.