Goto

Collaborating Authors

 Diagnosis


A Study on Mitigating Hard Boundaries of Decision-Tree-based Uncertainty Estimates for AI Models

arXiv.org Artificial Intelligence

Outcomes of data-driven AI models cannot be assumed to be always correct. To estimate the uncertainty in these outcomes, the uncertainty wrapper framework has been proposed, which considers uncertainties related to model fit, input quality, and scope compliance. Uncertainty wrappers use a decision tree approach to cluster input quality related uncertainties, assigning inputs strictly to distinct uncertainty clusters. Hence, a slight variation in only one feature may lead to a cluster assignment with a significantly different uncertainty. Our objective is to replace this with an approach that mitigates hard decision boundaries of these assignments while preserving interpretability, runtime complexity, and prediction performance. Five approaches were selected as candidates and integrated into the uncertainty wrapper framework. For the evaluation based on the Brier score, datasets for a pedestrian detection use case were generated using the CARLA simulator and YOLOv3. All integrated approaches achieved a softening, i.e., smoothing, of uncertainty estimation. Yet, compared to decision trees, they are not so easy to interpret and have higher runtime complexity. Moreover, some components of the Brier score impaired while others improved. Most promising regarding the Brier score were random forests. In conclusion, softening hard decision tree boundaries appears to be a trade-off decision.


ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions

arXiv.org Artificial Intelligence

One principal impediment in the successful deployment of AI-based Computer-Aided Diagnosis (CAD) systems in clinical workflows is their lack of transparent decision making. Although commonly used eXplainable AI methods provide some insight into opaque algorithms, such explanations are usually convoluted and not readily comprehensible except by highly trained experts. The explanation of decisions regarding the malignancy of skin lesions from dermoscopic images demands particular clarity, as the underlying medical problem definition is itself ambiguous. This work presents ExAID (Explainable AI for Dermatology), a novel framework for biomedical image analysis, providing multi-modal concept-based explanations consisting of easy-to-understand textual explanations supplemented by visual maps justifying the predictions. ExAID relies on Concept Activation Vectors to map human concepts to those learnt by arbitrary Deep Learning models in latent space, and Concept Localization Maps to highlight concepts in the input space. This identification of relevant concepts is then used to construct fine-grained textual explanations supplemented by concept-wise location information to provide comprehensive and coherent multi-modal explanations. All information is comprehensively presented in a diagnostic interface for use in clinical routines. An educational mode provides dataset-level explanation statistics and tools for data and model exploration to aid medical research and education. Through rigorous quantitative and qualitative evaluation of ExAID, we show the utility of multi-modal explanations for CAD-assisted scenarios even in case of wrong predictions. We believe that ExAID will provide dermatologists an effective screening tool that they both understand and trust. Moreover, it will be the basis for similar applications in other biomedical imaging fields.


Omitted Variable Bias in Machine Learned Causal Models

arXiv.org Machine Learning

We derive general, yet simple, sharp bounds on the size of the omitted variable bias for a broad class of causal parameters that can be identified as linear functionals of the conditional expectation function of the outcome. Such functionals encompass many of the traditional targets of investigation in causal inference studies, such as, for example, (weighted) average of potential outcomes, average treatment effects (including subgroup effects, such as the effect on the treated), (weighted) average derivatives, and policy effects from shifts in covariate distribution -- all for general, nonparametric causal models. Our construction relies on the Riesz-Frechet representation of the target functional. Specifically, we show how the bound on the bias depends only on the additional variation that the latent variables create both in the outcome and in the Riesz representer for the parameter of interest. Moreover, in many important cases (e.g, average treatment effects in partially linear models, or in nonseparable models with a binary treatment) the bound is shown to depend on two easily interpretable quantities: the nonparametric partial $R^2$ (Pearson's "correlation ratio") of the unobserved variables with the treatment and with the outcome. Therefore, simple plausibility judgments on the maximum explanatory power of omitted variables (in explaining treatment and outcome variation) are sufficient to place overall bounds on the size of the bias. Finally, leveraging debiased machine learning, we provide flexible and efficient statistical inference methods to estimate the components of the bounds that are identifiable from the observed distribution.


Artificial Intelligence Accurately Detects Fractures On X-rays, Alerts Human Readers

#artificialintelligence

A variety of readers were used to simulate real-life scenarios, including radiologists, orthopedic surgeons, emergency physicians and physician assistants, rheumatologists, and family physicians, all of whom read x-rays in real clinical practice to diagnose fractures in their patients. Each reader's diagnostic accuracy of fractures, with and without AI assistance, were compared against the gold standard. They also assessed the diagnostic performance of AI alone against the gold standard. AI assistance helped reduce missed fractures by 29% and increased readers' sensitivity by 16%, and by 30% for exams with more than 1 fracture, while improving specificity by 5%.


Reasoning About Causal Models With Infinitely Many Variables

arXiv.org Artificial Intelligence

Generalized structural equations models (GSEMs) [Peters and Halpern 2021], are, as the name suggests, a generalization of structural equations models (SEMs). They can deal with (among other things) infinitely many variables with infinite ranges, which is critical for capturing dynamical systems. We provide a sound and complete axiomatization of causal reasoning in GSEMs that is an extension of the sound and complete axiomatization provided by Halpern [2000] for SEMs. Considering GSEMs helps clarify what properties Halpern's axioms capture.


Fast Sparse Decision Tree Optimization via Reference Ensembles

arXiv.org Artificial Intelligence

Sparse decision tree optimization has been one of the most fundamental problems in AI since its inception and is a challenge at the core of interpretable machine learning. Sparse decision tree optimization is computationally hard, and despite steady effort since the 1960's, breakthroughs have only been made on the problem within the past few years, primarily on the problem of finding optimal sparse decision trees. However, current state-of-the-art algorithms often require impractical amounts of computation time and memory to find optimal or near-optimal trees for some real-world datasets, particularly those having several continuous-valued features. Given that the search spaces of these decision tree optimization problems are massive, can we practically hope to find a sparse decision tree that competes in accuracy with a black box machine learning model? We address this problem via smart guessing strategies that can be applied to any optimal branch-and-bound-based decision tree algorithm. We show that by using these guesses, we can reduce the run time by multiple orders of magnitude, while providing bounds on how far the resulting trees can deviate from the black box's accuracy and expressive power. Our approach enables guesses about how to bin continuous features, the size of the tree, and lower bounds on the error for the optimal decision tree. Our experiments show that in many cases we can rapidly construct sparse decision trees that match the accuracy of black box models. To summarize: when you are having trouble optimizing, just guess.


DECISION TREE IN A NUTSHELL

#artificialintelligence

When a bank considers whether it would offer a loan to someone or not, it considers a chronological list of questions to decide if it's safe to approve such a loan. The questions under consideration could begin with simple ones such as what's the individual's annual income. Based on the answers, the next set of questions could involve finding out if the person has any existing loans, has defaulted on credit card payments, etc. Assuming the person draws a salary of $30,000, has no existing loans or criminal record, and makes his credit card payments on time, the bank may offer him the loan. You can call this a basic form of a decision tree.


Machine Learning and the Challenge of Predicting Fake News

#artificialintelligence

Many Natural Language Processing (NLP) techniques exist for detecting "fake news". Multi-phase algorithms with Determined Decision Trees, Gradient Enlargement, and others have been used by various researchers and organizations with varying results. One study from researchers at Rensselaer Polytechnic Institute reported 83% accuracy in predicting whether a news article is from a reliable or unreliable source [1], while Facebook's 2019 attempt at developing an algorithm failed miserably, with some users experiencing a "maelstrom" of fake news [2]. A new study, published in the November 2021 issue of the Journal of Emerging Technologies and Innovative Research [3] performs an analysis of a wide range of AI models for efficacy, finding that models generally perform poorly, ranging from 60% to 77% accuracy. Separating fake news from real news is a challenge even for the most sophisticated AI. Simple content-related programs and shallow marking of the speech part (POS) fail to consider contextual information and are unable to accurately classify news stories as fact or fake unless combined with more sophisticated algorithms.


Intelligent Bearing Fault Diagnosis Method Combining Mixed Input and Hybrid CNN-MLP model

arXiv.org Artificial Intelligence

Rolling bearings are one of the most widely used bearings in industrial machines. Deterioration in the condition of rolling bearings can result in the total failure of rotating machinery. AI-based methods are widely applied in the diagnosis of rolling bearings. Hybrid NN-based methods have been shown to achieve the best diagnosis results. Typically, raw data is generated from accelerometers mounted on the machine housing. However, the diagnostic utility of each signal is highly dependent on the location of the corresponding accelerometer. This paper proposes a novel hybrid CNN-MLP model-based diagnostic method which combines mixed input to perform rolling bearing diagnostics. The method successfully detects and localizes bearing defects using acceleration data from a shaft-mounted wireless acceleration sensor. The experimental results show that the hybrid model is superior to the CNN and MLP models operating separately, and can deliver a high detection accuracy of 99,6% for the bearing faults compared to 98% for CNN and 81% for MLP models.


Applied Sciences

#artificialintelligence

In the fourth industrial revolution, or Industry 4.0, a key objective is to enhance equipment's ability to perceive its own health state and predict future behavior. The development of artificial intelligence, especially the progress made in deep learning, in the recent decade provides a promising tool in bolstering this enhancement. Such a tool can be a complement or alternative to conventional physics-based and signal-processing-based techniques in fault detection, diagnosis and prognosis applications. Researchers have started to build data-driven or hybrid models to further boost their prediction accuracy in the above applications, yet there are still some untouched or underexplored territories, such as causal inference, demystifying the black-box modelling, domain adaptation, automatic feature learning, etc. This special issue is to present current innovations and engineering achievements of scientists and industrial practitioners in the area of adopting artificial intelligence techniques in fault detection, diagnosis and prognosis.