Accuracy
4 Takeaways from 'How Google Does Machine Learning' course
Today, Machine Learning (ML) technology is simplified and abstracted to an API call so you can solve a data-intensive pattern matching problem easily. Google's Move Mirror is a great example. While creating a standalone ML-based consumer app is reasonably straightforward, it can be quite challenging to infuse ML at scale into a mission-critical enterprise-class cloud platform. Enterprise apps have to consider various steps in the machine learning life cycle including data cleansing, integration, and production deployment. Operationalizing ML is a topic by itself and I'll share more on that in a future post.
Predicting customer's gender and age depending on mobile phone data
AlZuabi, Ibrahim Mousa, Jafar, Assef, Aljoumaa, Kadan
In the age of data driven solution, the customer demographic attributes, such as gender and age, play a core role that may enable companies to enhance the offers of their services and target the right customer in the right time and place. In the marketing campaign, the companies want to target the real user of the GSM (global system for mobile communications), not the line owner. Where sometimes they may not be the same. This work proposes a method that predicts users' gender and age based on their behavior, services and contract information. We used call detail records (CDRs), customer relationship management (CRM) and billing information as a data source to analyze telecom customer behavior, and applied different types of machine learning algorithms to provide marketing campaigns with more accurate information about customer demographic attributes. This model is built using reliable data set of 18,000 users provided by SyriaTel Telecom Company, for training and testing. The model applied by using big data technology and achieved 85.6% accuracy in terms of user gender prediction and 65.5% of user age prediction. The main contribution of this work is the improvement in the accuracy in terms of user gender prediction and user age prediction based on mobile phone data and end-to-end solution that approaches customer data from multiple aspects in the telecom domain.
Inference of a Multi-Domain Machine Learning Model to Predict Mortality in Hospital Stays for Patients with Cancer upon Febrile Neutropenia Onset
Du, Xinsong, Min, Jae, Lemas, Dominick J., Prosperi, Mattia
Febrile neutropenia (FN) has been associated with high mortality, especially among adults with cancer. Understanding the patient and provider level heterogeneity in FN hospital admissions has potential to inform personalized interventions focused on increasing survival of individuals with FN. We leverage machine learning techniques to disentangling the complex interactions among multi domain risk factors in a population with FN. Data from the Healthcare Cost and Utilization Project (HCUP) National Inpatient Sample and Nationwide Inpatient Sample (NIS) were used to build machine learning based models of mortality for adult cancer patients who were diagnosed with FN during a hospital admission. In particular, the importance of risk factors from different domains (including demographic, clinical, and hospital associated information) was studied. A set of more interpretable (decision tree, logistic regression) as well as more black box (random forest, gradient boosting, neural networks) models were analyzed and compared via multiple cross validation. Our results demonstrate that a linear prediction score of FN mortality among adults with cancer, based on admission information is effective in classifying high risk patients; clinical diagnoses is the domain with the highest predictive power. A number of the risk variables (e.g. sepsis, kidney failure, etc.) identified in this study are clinically actionable and may inform future studies looking at the patients prior medical history are warranted.
Controlling false discoveries in large-scale experimentation: Challenges and solutions
"Scientific research has changed the world. Now it needs to change itself. There has been a growing concern about the validity of scientific findings. A multitude of journals, papers and reports have recognized the ever smaller number of replicable scientific studies. In 2016, one of the giants of scientific publishing, Nature, surveyed about 1,500 researchers across many different disciplines, asking for their stand on the status of reproducibility in their area of research. One of the many takeaways to the worrisome results of this survey is the following: 90% of the respondents agreed that there is a reproducibility crisis, and the overall top answer to boosting reproducibility was "better understanding of statistics". Indeed, many factors contributing to the explosion of irreproducible research stem from the neglect of the fact that statistics is no longer as static as it was in the first half of the 20th century, when statistical hypothesis testing came into prominence as a ...
Explaining precision and recall – Andreas Klintberg – Medium
The first days and weeks of getting into NLP, I had a hard time grasping the concepts of precision, recall and F1-score. Accuracy is also a metric which is tied to these, as well as micro-precision and macro-precision. These metrics are important in general machine learning and deep learning as well. However one of my colleagues (Thanks Marci!) explained it in an excellent way in which I'd thought I'd share. Many of you have probably already seen this, but for me it was a revelation in its simplicity.
Classifying textual data: shallow, deep and ensemble methods
Anderlucci, Laura, Guastadisegni, Lucia, Viroli, Cinzia
Nowadays the increasing and rapid progress of technology and the availability of electronic documents from a variety of sources have made a huge amount of textual data available. Hence, one of the prominent research topics of statistical andmachine learning communities is to provide suitable and feasible methods to extract high-quality information from unstructured textual data (Lata and Loar, 2018) for the different purposes of clustering, classification and document retrieval (Khan et al., 2010). This work originates from an empirical problem of classification of the content ofcalls made to the customer service of an important mobile phone company inItaly. The received calls are written down by an operator and classified into relevant classes (e.g.
Optimized data exploration applied to the simulation of a chemical process
Heese, Raoul, Walczak, Michal, Seidel, Tobias, Asprion, Norbert, Bortz, Michael
In complex simulation environments, certain parameter space regions may result in non-convergent or unphysical outcomes. All parameters can therefore be labeled with a binary class describing whether or not they lead to valid results. In general, it can be very difficult to determine feasible parameter regions, especially without previous knowledge. We propose a novel algorithm to explore such an unknown parameter space and improve its feasibility classification in an iterative way. Moreover, we include an additional optimization target in the algorithm to guide the exploration towards regions of interest and to improve the classification therein. In our method we make use of well-established concepts from the field of machine learning like kernel support vector machines and kernel ridge regression. From a comparison with a Kriging-based exploration approach based on recently published results we can show the advantages of our algorithm in a binary feasibility classification scenario with a discrete feasibility constraint violation. In this context, we also propose an improvement of the Kriging-based exploration approach. We apply our novel method to a fully realistic, industrially relevant chemical process simulation to demonstrate its practical usability and find a comparably good approximation of the data space topology from relatively few data points.
Mozilla to use machine learning to find code bugs before they ship
In a bid to cut the number of coding errors made in its Firefox browser, Mozilla is deploying Clever-Commit, a machine-learning-driven coding assistant developed in conjunction with game developer Ubisoft. Clever-Commit analyzes code changes as developers commit them to the Firefox codebase. It compares them to all the code it has seen before to see if they look similar to code that the system knows to be buggy. If the assistant thinks that a commit looks suspicious, it warns the developer. Presuming its analysis is correct, it means that the bug can be fixed before it gets committed into the source repository.
Controlling False Discoveries in Large-Scale Experimentation: Challenges and Solutions
"Scientific research has changed the world. Now it needs to change itself." There has been a growing concern about the validity of scientific findings. A multitude of journals, papers and reports have recognized the ever smaller number of replicable scientific studies. In 2016, one of the giants of scientific publishing, Nature, surveyed about 1,500 researchers across many different disciplines, asking for their stand on the status of reproducibility in their area of research.
Examining Adversarial Learning against Graph-based IoT Malware Detection Systems
Abusnaina, Ahmed, Khormali, Aminollah, Alasmary, Hisham, Park, Jeman, Anwar, Afsah, Meteriz, Ulku, Mohaisen, Aziz
The main goal of this study is to investigate the robustness of graph-based Deep Learning (DL) models used for Internet of Things (IoT) malware classification against Adversarial Learning (AL). We designed two approaches to craft adversarial IoT software, including Off-the-Shelf Adversarial Attack (OSAA) methods, using six different AL attack approaches, and Graph Embedding and Augmentation (GEA). The GEA approach aims to preserve the functionality and practicality of the generated adversarial sample through a careful embedding of a benign sample to a malicious one. Our evaluations demonstrate that OSAAs are able to achieve a misclassification rate (MR) of 100%. Moreover, we observed that the GEA approach is able to misclassify all IoT malware samples as benign.