Performance Analysis
Data-driven Design of Context-aware Monitors for Hazard Prediction in Artificial Pancreas Systems
Zhou, Xugui, Ahmed, Bulbul, Aylor, James H., Asare, Philip, Alemzadeh, Homa
Medical Cyber-physical Systems (MCPS) are vulnerable to accidental or malicious faults that can target their controllers and cause safety hazards and harm to patients. This paper proposes a combined model and data-driven approach for designing context-aware monitors that can detect early signs of hazards and mitigate them in MCPS. We present a framework for formal specification of unsafe system context using Signal Temporal Logic (STL) combined with an optimization method for patient-specific refinement of STL formulas based on real or simulated faulty data from the closed-loop system for the generation of monitor logic. We evaluate our approach in simulation using two state-of-the-art closed-loop Artificial Pancreas Systems (APS). The results show the context-aware monitor achieves up to 1.4 times increase in average hazard prediction accuracy (F1-score) over several baseline monitors, reduces false-positive and false-negative rates, and enables hazard mitigation with a 54% success rate while decreasing the average risk for patients.
A Conceptual Framework for Establishing Trust in Real World Intelligent Systems
Guckert, Michael, Gumpfer, Nils, Hannig, Jennifer, Keller, Till, Urquhart, Neil
Abstract: Intelligent information systems that contain emergent elements often encounter trust problems because results do not get sufficiently explained and the procedure itself can not be fully retraced. This is caused by a control flow depending either on stochastic elements or on the structure and relevance of the input data. Trust in such algorithms can be established by letting users interact with the system so that they can explore results and find patterns that can be compared with their expected solution. Reflecting features and patterns of human understanding of a domain against algorithmic results can create awareness of such patterns and may increase the trust that a user has in the solution. If expectations are not met, close inspection can be used to decide whether a solution conforms to the expectations or whether it goes beyond the expected. By either accepting or rejecting a solution, the user's set of expectations evolves and a learning process for the users is established. In this paper we present a conceptual framework that reflects and supports this process. The framework is the result of an analysis of two exemplary case studies from two different disciplines with information systems that assist experts in their complex tasks. Keywords: Intelligent Systems, AI, Trust, Explainable AI, Knowledge Management, Knowledge Patterns 1. INTRODUCTION uncommon and have been constructed in uncommon ways. Such techniques, a class to which systems that we now Human expertise in many aspects is largely based on call intelligent systems belong to, produce results of high prior knowledge and familiar patterns, which have either complexity (e.g.
Semi-Supervised Learning of Classifiers from a Statistical Perspective: A Brief Review
Ahfock, Daniel, McLachlan, Geoffrey J.
Due to the scarcity and often high acquisition cost of labelled data, machine learning methods that make effective use of large quantities of unlabelled data are being increasingly used. One such method is semi-supervised learning (SSL) where, in addition to labelled data, possibly large numbers of unlabelled observations are available at the time of the construction of the classification rule (classifier) to be used. Not surprisingly, semisupervised learning approaches have been gaining much attention in both the application oriented and the theoretical machine learning communities. However, theoretical analysis of SSL has so far been scarce. But last year, Ahfock and McLachlan (2020) provided an asymptotic basis on how to increase in certain situations the accuracy of the commonly used linear discriminant function formed from a partially classified sample as in SSL (Ahfock and McLachlan, 2020). The increase in accuracy can be of sufficient magnitude for this SSL-based classifier to have smaller error rate than that if it were formed from a completely classified sample.
Artificial Intelligence Methods Based Hierarchical Classification of Frontotemporal Dementia to Improve Diagnostic Predictability
Poonam, Km, Guha, Rajlakshmi, Chakrabarti, Partha P
Patients with Frontotemporal Dementia (FTD) have impaired cognitive abilities, executive and behavioral traits, loss of language ability, and decreased memory capabilities. Based on the distinct patterns of cortical atrophy and symptoms, the FTD spectrum primarily includes three variants: behavioral variant FTD (bvFTD), non-fluent variant primary progressive aphasia (nfvPPA), and semantic variant primary progressive aphasia (svPPA). The purpose of this study is to classify MRI images of every single subject into one of the spectrums of the FTD in a hierarchical order by applying data-driven techniques of Artificial Intelligence (AI) on cortical thickness data. This data is computed by FreeSurfer software. We used the Smallest Univalue Segment Assimilating Nucleus (SUSAN) technique to minimize the noise in cortical thickness data. Specifically, we took 204 subjects from the frontotemporal lobar degeneration neuroimaging initiative (NIFTD) database to validate this approach, and each subject was diagnosed in one of the diagnostic categories (bvFTD, svPPA, nfvPPA and cognitively normal). Our proposed automated classification model yielded classification accuracy of 86.5, 76, and 72.7 with support vector machine (SVM), linear discriminant analysis (LDA), and Naive Bayes methods, respectively, in 10-fold cross-validation analysis, which is a significant improvement on a traditional single multi-class model with an accuracy of 82.7, 73.4, and 69.2.
Parallel integrative learning for large-scale multi-response regression with incomplete outcomes
Dong, Ruipeng, Li, Daoji, Zheng, Zemin
Multi-task learning is increasingly used to investigate the association structure between multiple responses and a single set of predictor variables in many applications. In the era of big data, the coexistence of incomplete outcomes, large number of responses, and high dimensionality in predictors poses unprecedented challenges in estimation, prediction, and computation. In this paper, we propose a scalable and computationally efficient procedure, called PEER, for large-scale multi-response regression with incomplete outcomes, where both the numbers of responses and predictors can be high-dimensional. Motivated by sparse factor regression, we convert the multi-response regression into a set of univariate-response regressions, which can be efficiently implemented in parallel. Under some mild regularity conditions, we show that PEER enjoys nice sampling properties including consistency in estimation, prediction, and variable selection. Extensive simulation studies show that our proposal compares favorably with several existing methods in estimation accuracy, variable selection, and computation efficiency.
Supervised Feature Selection Techniques in Network Intrusion Detection: a Critical Review
Di Mauro, Mario, Galatro, Giovanni, Fortino, Giancarlo, Liotta, Antonio
Machine Learning (ML) techniques are becoming an invaluable support for network intrusion detection, especially in revealing anomalous flows, which often hide cyber-threats. Typically, ML algorithms are exploited to classify/recognize data traffic on the basis of statistical features such as inter-arrival times, packets length distribution, mean number of flows, etc. Dealing with the vast diversity and number of features that typically characterize data traffic is a hard problem. This results in the following issues: i) the presence of so many features leads to lengthy training processes (particularly when features are highly correlated), while prediction accuracy does not proportionally improve; ii) some of the features may introduce bias during the classification process, particularly those that have scarce relation with the data traffic to be classified. To this end, by reducing the feature space and retaining only the most significant features, Feature Selection (FS) becomes a crucial pre-processing step in network management and, specifically, for the purposes of network intrusion detection. In this review paper, we complement other surveys in multiple ways: i) evaluating more recent datasets (updated w.r.t. obsolete KDD 99) by means of a designed-from-scratch Python-based procedure; ii) providing a synopsis of most credited FS approaches in the field of intrusion detection, including Multi-Objective Evolutionary techniques; iii) assessing various experimental analyses such as feature correlation, time complexity, and performance. Our comparisons offer useful guidelines to network/security managers who are considering the incorporation of ML concepts into network intrusion detection, where trade-offs between performance and resource consumption are crucial.
How rotational invariance of common kernels prevents generalization in high dimensions
Donhauser, Konstantin, Wu, Mingqi, Yang, Fanny
Kernel ridge regression is well-known to achieve minimax optimal rates in low-dimensional settings. However, its behavior in high dimensions is much less understood. Recent work establishes consistency for kernel regression under certain assumptions on the ground truth function and the distribution of the input data. In this paper, we show that the rotational invariance property of commonly studied kernels (such as RBF, inner product kernels and fully-connected NTK of any depth) induces a bias towards low-degree polynomials in high dimensions. Our result implies a lower bound on the generalization error for a wide range of distributions and various choices of the scaling for kernels with different eigenvalue decays. This lower bound suggests that general consistency results for kernel ridge regression in high dimensions require a more refined analysis that depends on the structure of the kernel beyond its eigenvalue decay.
A conversation on artificial intelligence and gender bias
The world celebrated Women's History Month in March, and it is a timely moment for us to look at the forces that will shape gender parity in the future. Even as the pandemic accelerates digitization and the future of work, artificial intelligence (AI) stands out as a potentially helpful--or hurtful--tool in the equity agenda. McKinsey recorded a podcast in collaboration with Citi that dives into how gender bias is reflected in AI, why we must consciously debias our machine-human interfaces, and how AI can be a positive force for gender parity. Ioana Niculcea: Before we start the conversation, I think it's important for us to spend a moment assessing the amount of change that has taken place with regard to AI, and how the pace of that change has accelerated over the past few years. And many people argue that in light of the current COVID-19 circumstance, we'll feel further acceleration as people move toward digitization. I spent the past eight years in financial services, and it all started with data. Datafication of the industry was sort of the point of origin. And we hear often that over 90 percent of the data that we have today was created over the past two years. You hear things like every minute, there's over one million Facebook logins and 4.5 million YouTube videos being streamed, or 17,000 different Uber rides. There's a lot of data, and only 1 percent of that is being analyzed, as said today.
Study suggests that AI model selection might introduce bias
Register for a free or VIP pass today. The past several years have made it clear that AI and machine learning are not a panacea when it comes to fair outcomes. Applying algorithmic solutions to social problems can magnify biases against marginalized peoples; undersampling populations always results in worse predictive accuracy. But bias in AI doesn't arise from the datasets alone. Problem formulation, or the way researchers fit tasks to AI techniques, can contribute.
Handling Climate Change Using Counterfactuals: Using Counterfactuals in Data Augmentation to Predict Crop Growth in an Uncertain Climate Future
Temraz, Mohammed, Kenny, Eoin, Ruelle, Elodie, Shalloo, Laurence, Smyth, Barry, Keane, Mark T
Climate change poses a major challenge to humanity, especially in its impact on agriculture, a challenge that a responsible AI should meet. In this paper, we examine a CBR system (PBI-CBR) designed to aid sustainable dairy farming by supporting grassland management, through accurate crop growth prediction. As climate changes, PBI-CBR's historical cases become less useful in predicting future grass growth. Hence, we extend PBI-CBR using data augmentation, to specifically handle disruptive climate events, using a counterfactual method (from XAI). Study 1 shows that historical, extreme climate-events (climate outlier cases) tend to be used by PBI-CBR to predict grass growth during climate disrupted periods. Study 2 shows that synthetic outliers, generated as counterfactuals on a outlier-boundary, improve the predictive accuracy of PBI-CBR, during the drought of 2018. This study also shows that an instance-based counterfactual method does better than a benchmark, constraint-guided method.