Accuracy
E-MIIM: An Ensemble Learning based Context-Aware Mobile Telephony Model for Intelligent Interruption Management
Sarker, Iqbal H., Kayes, A. S. M., Furhad, Md Hasan, Islam, Mohammad Mainul, Islam, Md Shohidul
Nowadays, mobile telephony interruptions in our daily life activities are common because of the inappropriate ringing notifications of incoming phone calls in different contexts. Such interruptions may impact on the work attention not only for the mobile phone owners but also the surrounding people. Decision tree is the most popular machine learning classification technique that is used in existing context-aware mobile intelligent interruption management (MIIM) model to overcome such issues. However, a single decision tree based context-aware model may cause overfitting problem and thus decrease the prediction accuracy of the inferred model. Therefore, in this paper, we propose an ensemble machine learning based context-aware mobile telephony model for the purpose of intelligent interruption management by taking into account multi-dimensional contexts and name it "E-MIIM". The experimental results on individuals' real life mobile telephony datasets show that our E-MIIM model is more effective and outperforms existing MIIM model for predicting and managing individual's mobile telephony interruptions based on their relevant contextual information.
Predicting the Long-Term Outcomes of Biologics in Psoriasis Patients Using Machine Learning
Emam, Sepideh, Du, Amy X., Surmanowicz, Philip, Thomsen, Simon F., Greiner, Russ, Gniadecki, Robert
Background. Real-world data show that approximately 50% of psoriasis patients treated with a biologic agent will discontinue the drug because of loss of efficacy. History of previous therapy with another biologic, female sex and obesity were identified as predictors of drug discontinuations, but their individual predictive value is low. Objectives. To determine whether machine learning algorithms can produce models that can accurately predict outcomes of biologic therapy in psoriasis on individual patient level. Results. All tested machine learning algorithms could accurately predict the risk of drug discontinuation and its cause (e.g. lack of efficacy vs adverse event). The learned generalized linear model achieved diagnostic accuracy of 82%, requiring under 2 seconds per patient using the psoriasis patients dataset. Input optimization analysis established a profile of a patient who has best chances of long-term treatment success: biologic-naive patient under 49 years, early-onset plaque psoriasis without psoriatic arthritis, weight < 100 kg, and moderate-to-severe psoriasis activity (DLQI $\geq$ 16; PASI $\geq$ 10). Moreover, a different generalized linear model is used to predict the length of treatment for each patient with mean absolute error (MAE) of 4.5 months. However Pearson Correlation Coefficient indicates 0.935 linear dependencies between the actual treatment lengths and predicted ones. Conclusions. Machine learning algorithms predict the risk of drug discontinuation and treatment duration with accuracy exceeding 80%, based on a small set of predictive variables. This approach can be used as a decision-making tool, communicating expected outcomes to the patient, and development of evidence-based guidelines.
Is Data Privacy Real? Don't Bet on It - Knowledge@Wharton
In 2009, Netflix was sued for releasing movie ratings data from half a million subscribers who were identified only by unique ID numbers. The video streaming service divulged this "anonymized" information to the public as part of its Netflix Prize contest, in which participants were asked to use the data to develop a better content recommendation algorithm. But researchers from the University of Texas showed that as few as six movie ratings could be used to identify users. A closet lesbian sued Netflix, saying her anonymity was compromised. The lawsuit was settled in 2010. The Netflix case reveals a problem about which the public is just starting to learn, but that data analysts and computer scientists have known for years. In anonymized datasets where distinguishing characteristics of a person such as name and address have been deleted, even a handful of seemingly innocuous information can lead to identification.
Understanding Artificial Intelligence: A Comprehensive Glossary of Terms and Definitions Analytics Insight
It's not a matter of surprise that the world is moving ahead with fast pace due to marvels of artificial intelligence. The technology has added new values and innovation to our personal and professional lives. The sudden change can be daunting at times but on an optimistic edge the AI technology has complemented humankind with certain new aspects. It has given some new terms to our daily vocab which we haven't heard of before. Artificial Intelligence has also given new meanings to prevailing terms altogether.
Using machine learning and cheap satellite data to design rooftop solar power
This author's solar punk novel involves the team from Clean Coalition using their power grid maps, guiding business areas with strategic solar storage placement on a city level, taking into account Tesla's 1,600 superchargers, and everyone having solar storage in their homes. At some percentage, within this super distributed network we will gain resiliency. To get there will take patience, and smart tools. Researchers at the University of Massachusetts, Amherst campus, have built a software tool, called DeepRoof, which they say has achieved a "true positive rate" of 91.1% in identifying a roof's solar power potential, while using widely available (and cheap) satellite data from tools like Google Earth. Their goal in Deep Roof: a Data-Driven Approach For Solar Potential Estimation Using Rooftop Imagery, is to take a list of address (or GPS coordinates) from a contractor and hand back the solar power potential of those sites.
Estimation of preterm birth markers with U-Net segmentation network
Wลodarczyk, Tomasz, Pลotka, Szymon, Trzciลski, Tomasz, Rokita, Przemysลaw, Sochacki-Wรณjcicka, Nicole, Lipa, Michaล, Wรณjcicki, Jakub
Preterm birth is the most common cause of neonatal death. Current diagnostic methods that assess the risk of preterm birth involve the collection of maternal characteristics and transvaginal ultrasound imaging conducted in the first and second trimester of pregnancy. Analysis of the ultrasound data is based on visual inspection of images by gynaecologist, sometimes supported by hand-designed image features such as cervical length. Due to the complexity of this process and its subjective component, approximately 30% of spontaneous preterm deliveries are not correctly predicted. Moreover, 10% of the predicted preterm deliveries are false-positives. In this paper, we address the problem of predicting spontaneous preterm delivery using machine learning. To achieve this goal, we propose to first use a deep neural network architecture for segmenting prenatal ultrasound images and then automatically extract two biophysical ultrasound markers, cervical length (CL) and anterior cervical angle (ACA), from the resulting images. Our method allows to estimate ultrasound markers without human oversight. Furthermore, we show that CL and ACA markers, when combined, allow us to decrease false-negative ratio from 30% to 18%. Finally, contrary to the current approaches to diagnostics methods that rely only on gynaecologist's expertise, our method introduce objectively obtained results.
EPP: interpretable score of model predictive power
Gosiewska, Alicja, Bakala, Mateusz, Woznica, Katarzyna, Zwolinski, Maciej, Biecek, Przemyslaw
The most important part of model selection and hyperparameter tuning is the evaluation of model performance. The most popular measures, such as AUC, F1, ACC for binary classification, or RMSE, MAD for regression, or cross-entropy for multilabel classification share two common weaknesses. First is, that they are not on an interval scale. It means that the difference in performance for the two models has no direct interpretation. It makes no sense to compare such differences between datasets. Second is, that for k-fold cross-validation, the model performance is in most cases calculated as an average performance from particular folds, which neglects the information how stable is the performance for different folds. In this talk, we introduce a new EPP rating system for predictive models. We also demonstrate numerous advantages for this system, First, differences in EPP scores have probabilistic interpretation. Based on it we can assess the probability that one model will achieve better performance than another. Second, EPP scores can be directly compared between datasets. Third, they can be used for navigated hyperparameter tuning and model selection. Forth, we can create embeddings for datasets based on EPP scores.
Fairness Warnings and Fair-MAML: Learning Fairly with Minimal Data
Slack, Dylan, Friedler, Sorelle, Givental, Emile
In this paper, we advocate for the study of fairness techniques in low data situations. We propose two algorithms Fairness Warnings and Fair-MAML. The first is a model-agnostic algorithm that provides interpretable boundary conditions for when a fairly trained model may not behave fairly on similar but slightly different tasks within a given domain. The second is a fair meta-learning approach to train models that can be trained through gradient descent with the objective of "learning how to learn fairly". This method encodes more general notions of fairness and accuracy into the model so that it can learn new tasks within a domain both quickly and fairly from only a few training points. We demonstrate experimentally the individual utility of each model using relevant baselines for comparison and provide the first experiment to our knowledge of K-shot fairness, i.e. training a fair model on a new task with only K data points. Then, we illustrate the usefulness of both algorithms as a combined method for training models from a few data points on new tasks while using Fairness Warnings as interpretable boundary conditions under which the newly trained model may not be fair.
Identification of Pediatric Sepsis Subphenotypes for Enhanced Machine Learning Predictive Performance: A Latent Profile Analysis
Velez, Tom, Wang, Tony, Koutroulis, Ioannis, Chamberlain, James, Uppal, Amit, Yohannes, Seife, Tschampel, Tim, Apostolova, Emilia
Background: While machine learning (ML) models are rapidly emerging as promising screening tools in critical care medicine, the identification of homogeneous subphenotypes within populations with heterogeneous conditions such as pediatric sepsis may facilitate attainment of high-predictive performance of these prognostic algorithms. This study is aimed to identify subphenotypes of pediatric sepsis and demonstrate the potential value of partitioned data/subtyping-based training. Methods: This was a retrospective study of clinical data extracted from medical records of 6,446 pediatric patients that were admitted at a major hospital system in the DC area. Vitals and labs associated with patients meeting the diagnostic criteria for sepsis were used to perform latent profile analysis. Modern ML algorithms were used to explore the predictive performance benefits of reduced training data heterogeneity via label profiling. Results: In total 134 (2.1%) patients met the diagnostic criteria for sepsis in this cohort and latent profile analysis identified four profiles/subphenotypes of pediatric sepsis. Profiles 1 and 3 had the lowest mortality and included pediatric patients from different age groups. Profile 2 were characterized by respiratory dysfunction; profile 4 by neurological dysfunction and highest mortality rate (22.2%). Machine learning experiments comparing the predictive performance of models derived without training data profiling against profile targeted models suggest statistically significant improved performance of prediction can be obtained. For example, area under ROC curve (AUC) obtained to predict profile 4 with 24-hour data (AUC = .998, p < .0001) compared favorably with the AUC obtained from the model considering all profiles as a single homogeneous group (AUC = .918) with 24-hour data.
Bayesian Receiver Operating Characteristic Metric for Linear Classifiers
Hassan, Syeda Sakira, Huttunen, Heikki, Niemi, Jari, Tohka, Jussi
We propose a novel classifier accuracy metric: the Bayesian Area Under the Receiver Operating Characteristic Curve (CBAUC). The method estimates the area under the ROC curve and is related to the recently proposed Bayesian Error Estimator. The metric can assess the quality of a classifier using only the training dataset without the need for computationally expensive cross-validation. We derive a closed-form solution of the proposed accuracy metric for any linear binary classifier under the Gaussianity assumption, and study the accuracy of the proposed estimator using simulated and real-world data. These experiments confirm that the closed-form CBAUC is both faster and more accurate than conventional AUC estimators.