New Apache project Spot taps machine learning to sniff out cyber threats

The Apache Software Foundation is now incubating a project backed by Cloudera Inc. and Intel Corp. that aims to bolster cyber security with Big Data analytics and machine learning. Previously known as the Open Network Insights (ONI) initiative, the project is now called Apache Spot, since it uses machine learning to sniff out bad traffic amongst good data. It can also study network traffic to characterize its unique behavior using the open source distributed storage and processing software Hadoop, which helps it to discover if any anomalies are present. Intel launched the project back in February on Cloudera's cloud computing platform. Apache Spot works by storing large amounts of information within Hadoop, then using Apache Spark to process data from deep packet inspection of domain name system (DNS) traffic, connections and log files from proxies.