Performance Analysis
AI in Radiology: the Quest for the Killer App
We've just wrapped up our 2019 SIIM Annual Meeting in Denver during which AI was at the center of the discussions. I would like to reflect on two panel discussions I have interacted with on Wednesday 26th and Thursday 27th June in the Exhibition Hall Theater. The first one was about the economics of AI and the second about its current state in practice. I also had many interesting exchanges with key stakeholders of the AI and Radiology ecosystem ranging from Academia to Corporates. The panel included a diversified group of academic faculties, entrepreneurs and industry stakeholders including startups (Infervision, Ai.doc, Qure.ai โฆ) and established companies (Nuance, Blackford Analysis, Intelerad, Theracon, GE, Philips โฆ).
Fair-by-design explainable models for prediction of recidivism
Soares, Eduardo, Angelov, Plamen
Recidivism prediction provides decision makers with an assessment of the likelihood that a criminal defendant will reoffend that can be used in pre-trial decision-making. It can also be used for prediction of locations where crimes most occur, profiles that are more likely to commit violent crimes. While such instruments are gaining increasing popularity, their use is controversial as they may present potential discriminatory bias in the risk assessment. In this paper we propose a new fair-by-design approach to predict recidivism. It is prototype-based, learns locally and extracts empirically the data distribution. The results show that the proposed method is able to reduce the bias and provide human interpretable rules to assist specialists in the explanation of the given results.
Automated detection of oral pre-cancerous tongue lesions using deep learning for early diagnosis of oral cavity cancer
Shamim, Mohammed Zubair M., Syed, Sadatullah, Shiblee, Mohammad, Usman, Mohammed, Ali, Syed
Discovering oral cavity cancer (OCC) at an early stage is an effective way to increase patient survival rate. However, current initial screening process is done manually and is expensive for the average individual, especially in developing countries worldwide. This problem is further compounded due to the lack of specialists in such areas. Automating the initial screening process using artificial intelligence (AI) to detect pre-cancerous lesions can prove to be an effective and inexpensive technique that would allow patients to be triaged accordingly to receive appropriate clinical management. In this study, we have applied and evaluated the efficacy of six deep convolutional neural network (DCNN) models using transfer learning, for identifying pre-cancerous tongue lesions directly using a small data set of clinically annotated photographic images to diagnose early signs of OCC. DCNN model based on Vgg19 architecture was able to differentiate between benign and pre-cancerous tongue lesions with a mean classification accuracy of 0.98, sensitivity 0.89 and specificity 0.97. Additionally, the ResNet50 DCNN model was able to distinguish between five types of tongue lesions i.e. hairy tongue, fissured tongue, geographic tongue, strawberry tongue and oral hairy leukoplakia with a mean classification accuracy of 0.97. Preliminary results using an (AI+Physician) ensemble model demonstrate that an automated initial screening process of tongue lesions using DCNNs can achieve near-human level classification performance for diagnosing early signs of OCC in patients.
k-Relevance Vectors for Pattern Classification
Kassani, Peyman Hosseinzadeh, Kassani, Sara Hosseinzadeh
This study combines two different learning paradigms, k-nearest neighbor (k-NN) rule, as memory-based learning paradigm and relevance vector machines (RVM), as statistical learning paradigm. This combination is performed in kernel space and is called k-relevance vector (k-RV). The purpose is to improve the performance of k-NN rule. The proposed model significantly prunes irrelevant attributes. We also introduced a new parameter, responsible for early stopping of iterations in RVM. We show that the new parameter improves the classification accuracy of k-RV. Intensive experiments are conducted on several classification datasets from University of California Irvine (UCI) repository and two real datasets from computer vision domain. The performance of k-RV is highly competitive compared to a few state-of-the-arts in terms of classification accuracy.
Advancing subgroup fairness via sleeping experts
Blum, Avrim, Lykouris, Thodoris
We study methods for improving fairness to subgroups in settings with overlapping populations and sequential predictions. Classical notions of fairness focus on the balance of some property across different populations. However, in many applications the goal of the different groups is not to be predicted equally but rather to be predicted well. We demonstrate that the task of satisfying this guarantee for multiple overlapping groups is not straightforward and show that for the simple objective of unweighted average of false negative and false positive rate, satisfying this for overlapping populations can be statistically impossible even when we are provided predictors that perform well separately on each subgroup. On the positive side, we show that when individuals are equally important to the different groups they belong to, this goal is achievable; to do so, we draw a connection to the sleeping experts literature in online learning. Motivated by the one-sided feedback in natural settings of interest, we extend our results to such a feedback model. We also provide a game-theoretic interpretation of our results, examining the incentives of participants to join the system and to provide the system full information about predictors they may possess. We end with several interesting open problems concerning the strength of guarantees that can be achieved in a computationally efficient manner.
Evaluating Effects of Tuition Fees: Lasso for the Case of Germany
Gรถrgen, Konstantin, Schienle, Melanie
We study the effect of the introduction of university tuition fees on the enrollment behavior of students in Germany. For this, an appropriate Lasso-technique is crucial in order to identify the magnitude and significance of the effect due to potentially many relevant controlling factors and only a short time frame where fees existed. We show that a post-double selection strategy combined with stability selection determines a significant negative impact of fees on student enrollment and identifies relevant variables. This is in contrast to previous empirical studies and a plain linear panel regression which cannot detect any effect of tuition fees in this case. In our study, we explicitly deal with data challenges in the response variable in a transparent way and provide respective robust results. Moreover, we control for spatial cross-effects capturing the heterogeneity in the introduction scheme of fees across federal states ("Bundesl\"ander"), which can set their own educational policy. We also confirm the validity of our Lasso approach in a comprehensive simulation study.
Reduce Fraud and False Positives with Machine Learning
Balancing user experience with strong transaction and login protection is a constant battle for financial institutions โ one that can have tangible consequences for both bank and customer. Take, for example, User X. It's Cyber Monday, and User X wants to buy the latest smartphone on his favorite discount online store. The site has limited stock, so he logs in at 4am to ensure he'll be able to get a phone. He excitedly places the item in his cart, goes to check out, and clicks pay โ only to receive a message that the purchase was flagged as suspicious and blocked.
Many Heads Are Better Than One: The Case For Ensemble Learning
"The interests of truth require a diversity of opinions." Banks and lenders are increasingly turning to AI and machine learning to automate their core functions and make more accurate predictions in credit underwriting and fraud detection. ML practitioners can take advantage of a growing number of modeling algorithms, such as simple decision trees, random forests, gradient boosting machines, deep neural networks, and support vector machines. Each method has its strengths and weaknesses, which is why it often makes sense to combine ML algorithms to provide even greater predictive performance than any single ML method could provide on its own. This method of combining algorithms is known as ensembling.
FDA Clears GE Healthcare's AI Triage Algorithm on X-Ray Device
The US Food and Drug Administration (FDA) has cleared an artificial intelligence (AI) algorithm from GE Healthcare that analyzes chest x-rays for pneumothorax and helps flag suspected cases for radiologists to prioritize reading, the company announced today. The algorithm, part of a set of other quality-assurance algorithms named the Critical Care Suite, was developed to run on a GE Healthcare mobile x-ray device. The software is not yet for sale, and an outside expert expressed concern about its false positive rate. The idea for the application came from bedside clinician experience of waiting for radiologists to read chest x-rays, said Rachael Callcut, MD, MSPH, a surgeon and director of data science for the Center for Digital Health Innovation at the University of California, San Francisco. UCSF proposed developing the feature as part of a development partnership with GE Healthcare.
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review
Xu, Han, Ma, Yao, Liu, Haochen, Deb, Debayan, Liu, Hui, Tang, Jiliang, Jain, Anil
Deep neural networks (DNN) have achieved unprecedented success in numerous machine learning tasks in various domains. However, the existence of adversarial examples raises our concerns in adopting deep learning to safety-critical applications. As a result, we have witnessed increasing interests in studying attack and defense mechanisms for DNN models on different data types, such as images, graphs and text. Thus, it is necessary to provide a systematic and comprehensive overview of the main threats of attacks and the success of corresponding countermeasures. In this survey, we review the state of the art algorithms for generating adversarial examples and the countermeasures against adversarial examples, for three most popular data types, including images, graphs and text.