Statistical Learning
Programming Fairness in Algorithms
Being good is easy, what is difficult is being just. We need to defend the interests of those whom we've never met and never will. Note: This article is intended for a general audience to try and elucidate the complicated nature of unfairness in machine learning algorithms. As such, I have tried to explain concepts in an accessible way with minimal use of mathematics, in the hope that everyone can get something out of reading this. Supervised machine learning algorithms are inherently discriminatory. They are discriminatory in the sense that they use information embedded in the features of data to separate instances into distinct categories -- indeed, this is their designated purpose in life. This is reflected in the name for these algorithms which are often referred to as discriminative algorithms (splitting data into categories), in contrast to generative algorithms (generating data from a given category). When we use supervised machine learning, this "discrimination" is used as an aid to help us categorize our data into distinct categories within the data distribution, as illustrated below. Whilst this occurs when we apply discriminative algorithms -- such as support vector machines, forms of parametric regression (e.g. For example, using last week's weather data to try and predict the weather tomorrow has no moral valence attached to it.
Advancing from Predictive Maintenance to Intelligent Maintenance with AI and IIoT
Zheng, Haining, Paiva, Antonio R., Gurciullo, Chris S.
As Artificial Intelligent (AI) technology advances and increasingly large amounts of data become readily available via various Industrial Internet of Things (IIoT) projects, we evaluate the state of the art of predictive maintenance approaches and propose our innovative framework to improve the current practice. The paper first reviews the evolution of reliability modelling technology in the past 90 years and discusses major technologies developed in industry and academia. We then introduce the next generation maintenance framework - Intelligent Maintenance, and discuss its key components. This AI and IIoT based Intelligent Maintenance framework is composed of (1) latest machine learning algorithms including probabilistic reliability modelling with deep learning, (2) real-time data collection, transfer, and storage through wireless smart sensors, (3) Big Data technologies, (4) continuously integration and deployment of machine learning models, (5) mobile device and AR/VR applications for fast and better decision-making in the field. Particularly, we proposed a novel probabilistic deep learning reliability modelling approach and demonstrate it in the Turbofan Engine Degradation Dataset.
More is not Always Better: The Negative Impact of A-box Materialization on RDF2vec Knowledge Graph Embeddings
Iana, Andreea, Paulheim, Heiko
RDF2vec is an embedding technique for representing knowledge graph entities in a continuous vector space. In this paper, we investigate the effect of materializing implicit A-box axioms induced by subproperties, as well as symmetric and transitive properties. While it might be a reasonable assumption that such a materialization before computing embeddings might lead to better embeddings, we conduct a set of experiments on DBpedia which demonstrate that the materialization actually has a negative effect on the performance of RDF2vec. In our analysis, we argue that despite the huge body of work devoted on completing missing information in knowledge graphs, such missing implicit information is actually a signal, not a defect, and we show examples illustrating that assumption.
Continuous Artificial Prediction Markets as a Syndromic Surveillance Technique
According to the World Health Organisation (WHO) [World Health Organization, 2013], the United Nations directing and coordinating health authority, public health surveillance is: The continuous, systematic collection, analysis and interpretation of health-related data needed for the planning, implementation, and evaluation of public health practice. Public health surveillance practice has evolved over time. Although it was limited to pen and paper at the beginning of 20th century, it is now facilitated by huge advances in informatics. Information technology enhancements have changed the traditional approaches of capturing, storing, sharing and analysing of data and resulted efficient and reliable health surveillance techniques [Lombardo and Buckeridge, 2007]. The main objective and challenge of a health surveillance system is the earliest possible detection of a disease outbreak within a society for the purpose of protecting community health. In the past, before the widespread deployment of computers, health surveillance was based on reports received from medical care centres and laboratories.
Ethical Adversaries: Towards Mitigating Unfairness with Adversarial Machine Learning
Delobelle, Pieter, Temple, Paul, Perrouin, Gilles, Frénay, Benoît, Heymans, Patrick, Berendt, Bettina
Machine learning is being integrated into a growing number of critical systems with far-reaching impacts on society. Unexpected behaviour and unfair decision processes are coming under increasing scrutiny due to this widespread use and its theoretical considerations. Individuals, as well as organisations, notice, test, and criticize unfair results to hold model designers and deployers accountable. We offer a framework that assists these groups in mitigating unfair representations stemming from the training datasets. Our framework relies on two inter-operating adversaries to improve fairness. First, a model is trained with the goal of preventing the guessing of protected attributes' values while limiting utility losses. This first step optimizes the model's parameters for fairness. Second, the framework leverages evasion attacks from adversarial machine learning to generate new examples that will be misclassified. These new examples are then used to retrain and improve the model in the first step. These two steps are iteratively applied until a significant improvement in fairness is obtained. We evaluated our framework on well-studied datasets in the fairness literature -- including COMPAS -- where it can surpass other approaches concerning demographic parity, equality of opportunity and also the model's utility. We also illustrate our findings on the subtle difficulties when mitigating unfairness and highlight how our framework can assist model designers.
Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems
Nakka, Yashwanth Kumar, Liu, Anqi, Shi, Guanya, Anandkumar, Anima, Yue, Yisong, Chung, Soon-Jo
Learning-based control algorithms require data collection with abundant supervision for training. Safe exploration algorithms ensure the safety of this data collection process even when only partial knowledge is available. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained stochastic optimal control with dynamics learning and feedback control. We derive an iterative convex optimization algorithm that solves an \underline{Info}rmation-cost \underline{S}tochastic \underline{N}onlinear \underline{O}ptimal \underline{C}ontrol problem (Info-SNOC). The optimization objective encodes both optimal performance and exploration for learning, and the safety is incorporated as distributionally robust chance constraints. The dynamics are predicted from a robust regression model that is learned from data. The Info-SNOC algorithm is used to compute a sub-optimal pool of safe motion plans that aid in exploration for learning unknown residual dynamics under safety constraints. A stable feedback controller is used to execute the motion plan and collect data for model learning. We prove the safety of rollout from our exploration method and reduction in uncertainty over epochs, thereby guaranteeing the consistency of our learning method. We validate the effectiveness of Info-SNOC by designing and implementing a pool of safe trajectories for a planar robot. We demonstrate that our approach has higher success rate in ensuring safety when compared to a deterministic trajectory optimization approach.
Graph Embedding with Data Uncertainty
Laakom, Firas, Raitoharju, Jenni, Passalis, Nikolaos, Iosifidis, Alexandros, Gabbouj, Moncef
However, the impracticability of working in high dimensional spaces due to the curse of dimensionality and the realization that the data in many problems reside on manifolds with much lower dimensions than those of the original space, has led to the development of spectral-based subspace learning (SL) techniques. Spectral-based methods rely on the eigenanalysis of Scatter matrices. SL aims at determining a mapping of the original high-dimensional space into a lower-dimensional space preserving properties of interest in the input data. This mapping can be obtained using unsupervised methods, such as Principal Component Analysis (PCA) [1, 2], or supervised ones, such as Linear Discriminant Analysis (LDA) [3] and Marginal Fisher Analysis (MFA) [4]. Despite the different motivations of these spectral-based methods, a general formulation known as Graph Embedding was introduced in [4] to unify them within a common framework. For low-dimensional data, where dimensionality reduction is not needed and classification algorithms can be applied directly, many extensions modeling input data inaccuracies have recently been proposed [5, 6].
A Review of Emergency Incident Prediction, Resource Allocation and Dispatch Models
Mukhopadhyay, Ayan, Pettet, Geoffrey, Vazirizade, Sayyed, Lu, Di, Baroud, Hiba, Jaimes, Alex, Vorobeychik, Yevgeniy, Kochenderfer, Mykel, Dubey, Abhishek
Emergency response to incidents such as accidents, medical calls, and fires is one of the most pressing problems faced by communities across the globe. In the last fifty years, researchers have developed statistical, analytical, and algorithmic approaches for designing emergency response management (ERM) systems. In this survey, we present models for incident prediction, resource allocation, and dispatch for emergency incidents. We highlight the strengths and weaknesses of prior work in this domain and explore the similarities and differences between different modeling paradigms. Finally, we present future research directions. To the best of our knowledge, our work is the first comprehensive survey that explores the entirety of ERM systems.
Unsupervised Domain Adaptation with Progressive Adaptation of Subspaces
Unsupervised Domain Adaptation (UDA) aims to classify unlabeled target domain by transferring knowledge from labeled source domain with domain shift. Most of the existing UDA methods try to mitigate the adverse impact induced by the shift via reducing domain discrepancy. However, such approaches easily suffer a notorious mode collapse issue due to the lack of labels in target domain. Naturally, one of the effective ways to mitigate this issue is to reliably estimate the pseudo labels for target domain, which itself is hard. To overcome this, we propose a novel UDA method named Progressive Adaptation of Subspaces approach (PAS) in which we utilize such an intuition that appears much reasonable to gradually obtain reliable pseudo labels. Speci fically, we progressively and steadily refine the shared subspaces as bridge of knowledge transfer by adaptively anchoring/selecting and leveraging those target samples with reliable pseudo labels. Subsequently, the refined subspaces can in turn provide more reliable pseudo-labels of the target domain, making the mode collapse highly mitigated. Our thorough evaluation demonstrates that PAS is not only effective for common UDA, but also outperforms the state-of-the arts for more challenging Partial Domain Adaptation (PDA) situation, where the source label set subsumes the target one.
Machine Learning in Generation, Detection, and Mitigation of Cyberattacks in Smart Grid: A Survey
Haque, Nur Imtiazul, Shahriar, Md Hasan, Dastgir, Md Golam, Debnath, Anjan, Parvez, Imtiaz, Sarwat, Arif, Rahman, Mohammad Ashiqur
Smart grid (SG) is a complex cyber-physical system that utilizes modern cyber and physical equipment to run at an optimal operating point. Cyberattacks are the principal threats confronting the usage and advancement of the state-of-the-art systems. The advancement of SG has added a wide range of technologies, equipment, and tools to make the system more reliable, efficient, and cost-effective. Despite attaining these goals, the threat space for the adversarial attacks has also been expanded because of the extensive implementation of the cyber networks. Due to the promising computational and reasoning capability, machine learning (ML) is being used to exploit and defend the cyberattacks in SG by the attackers and system operators, respectively. In this paper, we perform a comprehensive summary of cyberattacks generation, detection, and mitigation schemes by reviewing state-of-the-art research in the SG domain. Additionally, we have summarized the current research in a structured way using tabular format. We also present the shortcomings of the existing works and possible future research direction based on our investigation.