An important task in exploring and analyzing real-world data sets is to detect unusual and interesting phenomena. In this paper, we study the group anomaly detection problem. Unlike traditional anomaly detection research that focuses on data points, our goal is to discover anomalous aggregated behaviors of groups of points. For this purpose, we propose the Flexible Genre Model (FGM). FGM is designed to characterize data groups at both the point level and the group level so as to detect various types of group anomalies. We evaluate the effectiveness of FGM on both synthetic and real data sets including images and turbulence data, and show that it is superior to existing approaches in detecting group anomalies.
AnomalyDetection is an open-source R package to detect anomalies which is robust, from a statistical standpoint, in the presence of seasonality and an underlying trend. The AnomalyDetection package can be used in wide variety of contexts. The underlying algorithm – referred to as Seasonal Hybrid ESD (S-H-ESD) builds upon the Generalized ESD test for detecting anomalies. Note that S-H-ESD can be used to detect both global as well as local anomalies. This is achieved by employing time series decomposition and using robust statistical metrics, viz., median together with ESD.
Simple enough to be embedded in text as a sparkline, but able to speak volumes about your business, time series data is the basic input of Anodot's automated anomaly detection system. This article begins our three-part series in which we take a closer look at the specific techniques Anodot uses to extract insights from your data.
The progress of the Internet of Things (IoT) and its industrial usage has been nothing short of amazing. The IoT as we know it has been aided through the help of organizations owning up the process and implementing the results to garner a more efficient system. Since I have been associated with Big Data, Machine Learning, and Data Science over an extensive period of time, the implications in IoT and how most industrial giants need external assistance is nothing new for me. In fact, you can take a look at the IoT sphere of today through an eye as experienced as mine, and you will be able to gauge what the implication is all about it.
System states that are anomalous from the perspective of a domain expert occur frequently in some anomaly detection problems. The performance of commonly used unsupervised anomaly detection methods may suffer in that setting, because they use frequency as a proxy for anomaly. We propose a novel concept for anomaly detection, called relative anomaly detection. It is tailored to be robust towards anomalies that occur frequently, by taking into account their location relative to the most typical observations. The approaches we develop are computationally feasible even for large data sets, and they allow real-time detection. We illustrate using data sets of potential scraping attempts and Wi-Fi channel utilization, both from Google, Inc.