The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider Machine Learning

We describe the outcome of a data challenge conducted as part of the Dark Machines Initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims at detecting signals of new physics at the LHC using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 Billion simulated LHC events corresponding to $10~\rm{fb}^{-1}$ of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at Code to reproduce the analysis is provided at

Local Trend Inconsistency: A Prediction-driven Approach to Unsupervised Anomaly Detection in Multi-seasonal Time Series Machine Learning

Abstract--Online detection of anomalies in time series is a key technique in various event-sensitive scenarios such a s robotic system monitoring, smart sensor networks and data center security. However, the increasing diversity of data sources and demands are making this task more challenging than ever . First, the rapid increase of unlabeled data makes supervise d learning no longer suitable in many cases. Second, a great po rtion of time series have complex seasonality features. Third, on -line anomaly detection needs to be fast and reliable. In view of this, we in this paper adopt an unsupervised prediction-dri ven approach on the basis of a backbone model combining a series decomposition part and an inference part. We then propose a novel metric, Local Trend Inconsistency (L TI), along with a detection algorithm that efficiently computes L TI chronolo gically along the series and marks each data point with a score indica ting its probability of being anomalous. The result shows that our scheme outperforms several representative anomaly detection alg orithms in Area Under Curve (AUC) metric with decent time efficiency. While time series data has been ubiquitous before the coming of big data era, a large number of recently emerging technical scenarios like autonomous driving, edge computi ng and Internet of Things (IoT) pose new challenges to the detection of anomalies in this type of data. In the meantime, detection techniques that can provide early, reliable repo rts of anomaly has become crucial for a wide range of systems requiring 24/7 monitoring services. In cloud data centers, for example, a distributed monitoring system usually collects a variety of log data from virtual machine level to cluster lev el on a regular basis and sends them to a central detection module, which needs to analyze the aggregated time series to detect any anomalous events including hardware breakdown, unavailable services and cyber attacks. This requires an on - line detector capable of making reliable detections (i.e., with strong sensitivity and specificity), otherwise it could bri ng about unnecessary cost of maintenance.

Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series Machine Learning

Today's Cyber-Physical Systems (CPSs) are large, complex, and affixed with networked sensors and actuators that are targets for cyber-attacks. Conventional detection techniques are unable to deal with the increasingly dynamic and complex nature of the CPSs. On the other hand, the networked sensors and actuators generate large amounts of data streams that can be continuously monitored for intrusion events. Unsupervised machine learning techniques can be used to model the system behaviour and classify deviant behaviours as possible attacks. In this work, we proposed a novel Generative Adversarial Networks-based Anomaly Detection (GAN-AD) method for such complex networked CPSs. We used LSTM-RNN in our GAN to capture the distribution of the multivariate time series of the sensors and actuators under normal working conditions of a CPS. Instead of treating each sensor's and actuator's time series independently, we model the time series of multiple sensors and actuators in the CPS concurrently to take into account of potential latent interactions between them. To exploit both the generator and the discriminator of our GAN, we deployed the GAN-trained discriminator together with the residuals between generator-reconstructed data and the actual samples to detect possible anomalies in the complex CPS. We used our GAN-AD to distinguish abnormal attacked situations from normal working conditions for a complex six-stage Secure Water Treatment (SWaT) system. Experimental results showed that the proposed strategy is effective in identifying anomalies caused by various attacks with high detection rate and low false positive rate as compared to existing methods.