Accuracy
Autoencoder based Anomaly Detection and Explained Fault Localization in Industrial Cooling Systems
Holly, Stephanie, Heel, Robin, Katic, Denis, Schoeffl, Leopold, Stiftinger, Andreas, Holzner, Peter, Kaufmann, Thomas, Haslhofer, Bernhard, Schall, Daniel, Heitzinger, Clemens, Kemnitz, Jana
Anomaly detection in large industrial cooling systems is very challenging due to the high data dimensionality, inconsistent sensor recordings, and lack of labels. The state of the art for automated anomaly detection in these systems typically relies on expert knowledge and thresholds. However, data is viewed isolated and complex, multivariate relationships are neglected. In this work, we present an autoencoder based end-to-end workflow for anomaly detection suitable for multivariate time series data in large industrial cooling systems, including explained fault localization and root cause analysis based on expert knowledge. We identify system failures using a threshold on the total reconstruction error (autoencoder reconstruction error including all sensor signals). For fault localization, we compute the individual reconstruction error (autoencoder reconstruction error for each sensor signal) allowing us to identify the signals that contribute most to the total reconstruction error. Expert knowledge is provided via look-up table enabling root-cause analysis and assignment to the affected subsystem. We demonstrated our findings in a cooling system unit including 34 sensors over a 8-months time period using 4-fold cross validation approaches and automatically created labels based on thresholds provided by domain experts. Using 4-fold cross validation, we reached a F1-score of 0.56, whereas the autoencoder results showed a higher consistency score (CS of 0.92) compared to the automatically created labels (CS of 0.62) -- indicating that the anomaly is recognized in a very stable manner. The main anomaly was found by the autoencoder and automatically created labels and was also recorded in the log files. Further, the explained fault localization highlighted the most affected component for the main anomaly in a very consistent manner.
Comparison of different automatic solutions for resection cavity segmentation in postoperative MRI volumes including longitudinal acquisitions
Canalini, Luca, Klein, Jan, de Barros, Nuno Pedrosa, Sima, Diana Maria, Miller, Dorothea, Hahn, Horst
In this work, we compare five deep learning solutions to automatically segment the resection cavity in postoperative MRI. The proposed methods are based on the same 3D U-Net architecture. We use a dataset of postoperative MRI volumes, each including four MRI sequences and the ground truth of the corresponding resection cavity. Four solutions are trained with a different MRI sequence. Besides, a method designed with all the available sequences is also presented. Our experiments show that the method trained only with the T1 weighted contrast-enhanced MRI sequence achieves the best results, with a median DICE index of 0.81.
Generative Adversarial Learning for Trusted and Secure Clustering in Industrial Wireless Sensor Networks
Yang, Liu, Yang, Simon X., Li, Yun, Lu, Yinzhi, Guo, Tan
Traditional machine learning techniques have been widely used to establish the trust management systems. However, the scale of training dataset can significantly affect the security performances of the systems, while it is a great challenge to detect malicious nodes due to the absence of labeled data regarding novel attacks. To address this issue, this paper presents a generative adversarial network (GAN) based trust management mechanism for Industrial Wireless Sensor Networks (IWSNs). First, type-2 fuzzy logic is adopted to evaluate the reputation of sensor nodes while alleviating the uncertainty problem. Then, trust vectors are collected to train a GAN-based codec structure, which is used for further malicious node detection. Moreover, to avoid normal nodes being isolated from the network permanently due to error detections, a GAN-based trust redemption model is constructed to enhance the resilience of trust management. Based on the latest detection results, a trust model update method is developed to adapt to the dynamic industrial environment. The proposed trust management mechanism is finally applied to secure clustering for reliable and real-time data transmission, and simulation results show that it achieves a high detection rate up to 96%, as well as a low false positive rate below 8%.
One Graph to Rule them All: Using NLP and Graph Neural Networks to analyse Tolkien's Legendarium
Perri, Vincenzo, Qarkaxhija, Lisi, Zehe, Albin, Hotho, Andreas, Scholtes, Ingo
Natural Language Processing and Machine Learning have considerably advanced Computational Literary Studies. Similarly, the construction of co-occurrence networks of literary characters, and their analysis using methods from social network analysis and network science, have provided insights into the micro- and macro-level structure of literary texts. Combining these perspectives, in this work we study character networks extracted from a text corpus of J.R.R. Tolkien's Legendarium. We show that this perspective helps us to analyse and visualise the narrative style that characterises Tolkien's works. Addressing character classification, embedding and co-occurrence prediction, we further investigate the advantages of state-of-the-art Graph Neural Networks over a popular word embedding method. Our results highlight the large potential of graph learning in Computational Literary Studies.
How Wish A/B tests percentiles
And Jᵢ is the number of page loading times lower than the sample P95 for user i in one bucket. Note that all of these summary statistics are aggregated at the experiment bucket level, making the data pipeline much simpler. With regard to the tuning parameter, we found 0.25 led to satisfactory results for percentiles like P50 and P95. It may require more careful tuning when it comes to extreme percentiles like P99 or P99.9.
Classification Metrics Walkthrough: Logistic Regression with Accuracy, Precision, Recall, and ROC - KDnuggets
Metrics are an important element of machine learning. In regard to classification tasks, there are different types of metrics that allow you to assess the performance of machine learning models. However, it can be difficult to choose the right one for your task at hand. In this article, I will be going through 4 common classification metrics: Accuracy, Precision, Recall, and ROC in relation to Logistic Regression. Logistic Regression is a form of Supervised Learning - when the algorithm learns on a labeled dataset and analyses the training data.
Train a Custom Object Detector with Detectron2 and FiftyOne
In recent years, every aspect of the Machine Learning (ML) lifecycle has had tooling developed to make it easier to bring a custom model from an idea to a reality. The most exciting part is that the community has a propensity for open-source tools, like Pytorch and Tensorflow, allowing the model development process to be more transparent and replicable. In this post, we take a look at how to integrate two open-source tools tackling different parts of an ML project: FiftyOne and Detectron2. Detectron2 is a library developed by Facebook AI Research designed to allow you to easily train state-of-the-art detection and segmentation algorithms on your own data. FiftyOne is a toolkit designed to let you easily visualize your data, curate high-quality datasets, and analyze your model results.
A Unique Way of Visualising Confusion Matrix -- Sankey Chart
Originally published on Towards AI the World's Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses. However, when communicating with non-technical stakeholders, the confusion matrix might seem unintuitive .
Metaphorical Paraphrase Generation: Feeding Metaphorical Language Models with Literal Texts
Ottolina, Giorgio, Pavlopoulos, John
This study presents a new approach to metaphorical paraphrase generation by masking literal tokens of literal sentences and unmasking them with metaphorical language models. Unlike similar studies, the proposed algorithm does not only focus on verbs but also on nouns and adjectives. Despite the fact that the transfer rate for the former is the highest (56%), the transfer of the latter is feasible (24% and 31%). Human evaluation showed that our system-generated metaphors are considered more creative and metaphorical than human-generated ones while when using our transferred metaphors for data augmentation improves the state of the art in metaphorical sentence classification by 3% in F1.
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
Yang, Jingkang, Wang, Pengyun, Zou, Dejian, Zhou, Zitang, Ding, Kunyuan, Peng, Wenxuan, Wang, Haoqi, Chen, Guangyao, Li, Bo, Sun, Yiyou, Du, Xuefeng, Zhou, Kaiyang, Zhang, Wayne, Hendrycks, Dan, Li, Yixuan, Liu, Ziwei
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature. However, the field currently lacks a unified, strictly formulated, and comprehensive benchmark, which often results in unfair comparisons and inconclusive results. From the problem setting perspective, OOD detection is closely related to neighboring fields including anomaly detection (AD), open set recognition (OSR), and model uncertainty, since methods developed for one domain are often applicable to each other. To help the community to improve the evaluation and advance, we build a unified, well-structured codebase called OpenOOD, which implements over 30 methods developed in relevant fields and provides a comprehensive benchmark under the recently proposed generalized OOD detection framework. With a comprehensive comparison of these methods, we are gratified that the field has progressed significantly over the past few years, where both preprocessing methods and the orthogonal post-hoc methods show strong potential. We invite readers to use our OpenOOD codebase to develop and contribute. The full experimental results are available in this table.