Goto

Collaborating Authors

 real-time anomaly detection


A real-time anomaly detection method for robots based on a flexible and sparse latent space

arXiv.org Artificial Intelligence

The growing demand for robots to operate effectively in diverse environments necessitates the need for robust real-time anomaly detection techniques during robotic operations. However, deep learning-based models in robotics face significant challenges due to limited training data and highly noisy signal features. In this paper, we present Sparse Masked Autoregressive Flow-based Adversarial AutoEncoder model to address these problems. This approach integrates Masked Autoregressive Flow model into Adversarial AutoEncoders to construct a flexible latent space and utilize Sparse autoencoder to efficiently focus on important features, even in scenarios with limited feature space. Our experiments demonstrate that the proposed model achieves a 4.96% to 9.75% higher area under the receiver operating characteristic curve for pick-and-place robotic operations with randomly placed cans, compared to existing state-of-the-art methods. Notably, it showed up to 19.67% better performance in scenarios involving collisions with lightweight objects. Additionally, unlike the existing state-of-the-art model, our model performs inferences within 1 millisecond, ensuring real-time anomaly detection. These capabilities make our model highly applicable to machine learning-based robotic safety systems in dynamic environments. The code is available at https://github.com/twkang43/sparse-maf-aae.


Adaptive State-Space Mamba for Real-Time Sensor Data Anomaly Detection

arXiv.org Artificial Intelligence

Modern sensor networks generate large volumes of sequential data in real time. Detecting unusual patterns or anomalies within these streams is crucial for numerous applications, ranging from industrial process monitoring to environmental surveillance and predictive maintenance. Traditional anomaly detection approaches often rely on static feature engineering or rigid statistical assumptions, limiting their applicability in dynamic environments. Recently, deep learning models have emerged as powerful alternatives for sequence modeling, with architectures such as recurrent neural networks (RNNs) and transformers [4] achieving impressive performance. Nevertheless, these methods can be resource-intensive for streaming data scenarios, where real-time or near-real-time processing is essential. State-space models (SSMs) offer a principled approach to describing the evolution of a hidden state as a function of inputs and outputs [1]. While these models have been actively studied for signal processing and time-series forecasting, they have gained traction in broader AI tasks as well. For instance, Wang and Liu [5] introduced a novel usage of a state-space model (called "Mamba") for efficient text-driven image style transfer. Inspired by the notion of modeling sequential dependencies via hidden states, we adapt the core Mamba idea to the entirely different application of real-time anomaly detection in sensor data.


A Novel Zero-Touch, Zero-Trust, AI/ML Enablement Framework for IoT Network Security

arXiv.org Artificial Intelligence

The IoT facilitates a connected, intelligent, and sustainable society; therefore, it is imperative to protect the IoT ecosystem. The IoT-based 5G and 6G will leverage the use of machine learning and artificial intelligence (ML/AI) more to pave the way for autonomous and collaborative secure IoT networks. Zero-touch, zero-trust IoT security with AI and machine learning (ML) enablement frameworks offers a powerful approach to securing the expanding landscape of Internet of Things (IoT) devices. This paper presents a novel framework based on the integration of Zero Trust, Zero Touch, and AI/ML powered for the detection, mitigation, and prevention of DDoS attacks in modern IoT ecosystems. The focus will be on the new integrated framework by establishing zero trust for all IoT traffic, fixed and mobile 5G/6G IoT network traffic, and data security (quarantine-zero touch and dynamic policy enforcement). We perform a comparative analysis of five machine learning models, namely, XGBoost, Random Forest, K-Nearest Neighbors, Stochastic Gradient Descent, and Native Bayes, by comparing these models based on accuracy, precision, recall, F1-score, and ROC-AUC. Results show that the best performance in detecting and mitigating different DDoS vectors comes from the ensemble-based approaches.


Real-time Anomaly Detection at the L1 Trigger of CMS Experiment

arXiv.org Artificial Intelligence

The Compact Muon Solenoid (CMS) experiment studies these collisions to uncover potential Beyond Standard Model (BSM) physics and precisely measure rare Standard Model (SM) processes [2]. While the high collision rate at the LHC increases the probability of producing and detecting rare processes, the nearly 100 million channels of the CMS detector also generate an enormous amount of data [10, 14]. Only a small fraction of the 40 MHz proton-proton collision events--around 1,000 per second--can be stored for detailed offline analysis. To meet this stringent data reduction, events are selected using a two-tiered trigger system. The first level (L1), composed of custom hardware processors built with field-programmable gate arrays (FPGAs), uses information from the calorimeters and muon detectors to select events at a rate of around 100 kHz within a fixed latency of 4 [14]. The second level, the high-level trigger (HLT), consists of a processor farm running optimized event reconstruction software, reducing the rate to around 1 kHz before storage [10].


A Real-time Anomaly Detection Using Convolutional Autoencoder with Dynamic Threshold

arXiv.org Artificial Intelligence

The majority of modern consumer-level energy is generated by real-time smart metering systems. These frequently contain anomalies, which prevent reliable estimates of the series' evolution. This work introduces a hybrid modeling approach combining statistics and a Convolutional Autoencoder with a dynamic threshold. The threshold is determined based on Mahalanobis distance and moving averages. It has been tested using real-life energy consumption data collected from smart metering systems. The solution includes a real-time, meter-level anomaly detection system that connects to an advanced monitoring system. This makes a substantial contribution by detecting unusual data movements and delivering an early warning. Early detection and subsequent troubleshooting can financially benefit organizations and consumers and prevent disasters from occurring.


A hybrid IndRNNLSTM approach for real-time anomaly detection in software-defined networks

arXiv.org Artificial Intelligence

Anomaly detection in SDN using data flow prediction is a difficult task. This problem is included in the category of time series and regression problems. Machine learning approaches are challenging in this field due to the manual selection of features. On the other hand, deep learning approaches have important features due to the automatic selection of features. Meanwhile, RNN-based approaches have been used the most. The LSTM and GRU approaches learn dependent entities well; on the other hand, the IndRNN approach learns non-dependent entities in time series. The proposed approach tried to use a combination of IndRNN and LSTM approaches to learn dependent and non-dependent features. Feature selection approaches also provide a suitable view of features for the models; for this purpose, four feature selection models, Filter, Wrapper, Embedded, and Autoencoder were used. The proposed IndRNNLSTM algorithm, in combination with Embedded, was able to achieve MAE=1.22 and RMSE=9.92 on NSL-KDD data.


Neuro-symbolic Empowered Denoising Diffusion Probabilistic Models for Real-time Anomaly Detection in Industry 4.0

arXiv.org Artificial Intelligence

Industry 4.0 involves the integration of digital technologies, such as IoT, Big Data, and AI, into manufacturing and industrial processes to increase efficiency and productivity. As these technologies become more interconnected and interdependent, Industry 4.0 systems become more complex, which brings the difficulty of identifying and stopping anomalies that may cause disturbances in the manufacturing process. This paper aims to propose a diffusion-based model for real-time anomaly prediction in Industry 4.0 processes. Using a neuro-symbolic approach, we integrate industrial ontologies in the model, thereby adding formal knowledge on smart manufacturing. Finally, we propose a simple yet effective way of distilling diffusion models through Random Fourier Features for deployment on an embedded system for direct integration into the manufacturing process. To the best of our knowledge, this approach has never been explored before.


Machine Learning for Real-Time Anomaly Detection in Optical Networks

arXiv.org Artificial Intelligence

This work proposes a real-time anomaly detection scheme that leverages the multi-step ahead prediction capabilities of encoder-decoder (ED) deep learning models with recurrent units. Specifically, an encoder-decoder is used to model soft-failure evolution over a long future horizon (i.e., for several days ahead) by analyzing past quality-of-transmission (QoT) observations. This information is subsequently used for real-time anomaly detection (e.g., of attack incidents), as the knowledge of how the QoT is expected to evolve allows capturing unexpected network behavior. Specifically, for anomaly detection, a statistical hypothesis testing scheme is used, alleviating the limitations of supervised (SL) and unsupervised learning (UL) schemes, usually applied for this purpose. Indicatively, the proposed scheme eliminates the need for labeled anomalies, required when SL is applied, and the need for on-line analyzing entire datasets to identify abnormal instances (i.e., UL). Overall, it is shown that by utilizing QoT evolution information, the proposed approach can effectively detect abnormal deviations in real-time. Importantly, it is shown that the information concerning soft-failure evolution (i.e., QoT predictions) is essential to accurately detect anomalies.


DSC Webinar Series: AI in Action: Real-time Anomaly Detection

#artificialintelligence

Artificial intelligence is no longer in the future. You will learn how to: Detect anomalies in IoT applications using TIBCO Data Science with deep learning libraries (e.g. H2O, Python, TensorFlow, Amazon SageMaker) Use TIBCO Data Science models on the AWS Marketplace Deploy models into operations for real-time monitoring and surveillance Optimize your business and experience explosive growth with real-time anomaly detection.


Why Real-Time, AI-Based Anomaly Detection Is a No-Brainer - DZone AI

#artificialintelligence

In the earliest days of big data, collection was the top priority. Business leaders needed to find innovative ways to collect as much information about customers and operations as possible. Now that this goal has been accomplished, a new problem has arisen. There is enough data available to optimize user experience, network performance, business operations, and more, however, between 60 and 73 percent of that data never gets put to good use. There is an overwhelming amount of different metrics and systems to track, making it increasingly difficult to evaluate business patterns and, more importantly, deviations. This is why anomaly detection plays such a critical role in the modern, efficient enterprise.