This paper considers the learning of Boolean rules in either disjunctive normal form (DNF, OR-of-ANDs, equivalent to decision rule sets) or conjunctive normal form (CNF, AND-of-ORs) as an interpretable model for classification. An integer program is formulated to optimally trade classification accuracy for rule simplicity. Column generation (CG) is used to efficiently search over an exponential number of candidate clauses (conjunctions or disjunctions) without the need for heuristic rule mining. This approach also bounds the gap between the selected rule set and the best possible rule set on the training data. To handle large datasets, we propose an approximate CG algorithm using randomization. Compared to three recently proposed alternatives, the CG algorithm dominates the accuracy-simplicity trade-off in 8 out of 16 datasets. When maximized for accuracy, CG is competitive with rule learners designed for this purpose, sometimes finding significantly simpler solutions that are no less accurate.

Developers know a lot about the machine learning (ML) systems they create and manage, that's a given. However, there is a need for non-developers to have a high level understanding of the types of systems. Artificial neural networks and expert systems are the classical two key classes. With the advanced in computing performance, software capabilities and algorithm complexity, analytical algorithm can arguably be said to have joined the other two. This article is an overview of the three types.

Dr. Akli Adjaoute, CEO of Brighterion, wrote this AI-focused piece as part of our 2018 year-end eBook. On Dec. 3, 2018, the U.S. Treasury's FinCEN and Federal Banking agencies issued a joint statement encouraging innovative industry approaches to combating money laundering, terrorist financing and other illicit financial threats. As a result, anti-money laundering (AML) has been occupying the headlines as of late. The financial industry has paid $321 billion in fines just through the end of last year, as estimated by Boston Consulting Group. JPMorgan had to pay more than $2 billion in fines due to violation of the Bank Secrecy Act, tied in part to the infamous Bernie Madoff scheme.

I have been wanting to start cross-selling at my family's webshop Seniorshoppen.dk for a while. The problem is that we sell 500 different products, so I have been pushing it off as it would take a lot of time to manually select possible cross-sells for each product we sell. Therefore, I have been thinking a lot about how to automate this best using machine learning or other techniques, I just haven't got around to doing it. For Black Friday we decided to require our customers to buy a minimum of two products to get a 25% discount. So, it was a natural time to actually execute on this to "help" our customers choose a second product for their Black Friday shopping spree. This is how I did it.

Data extracted from software repositories is used intensively in Software Engineering research, for example, to predict defects in source code. In our research in this area, with data from open source projects as well as an industrial partner, we noticed several shortcomings of conventional data mining approaches for classification problems: (1) Domain experts' acceptance is of critical importance, and domain experts can provide valuable input, but it is hard to use this feedback. (2) The evaluation of the model is not a simple matter of calculating AUC or accuracy. Instead, there are multiple objectives of varying importance, but their importance cannot be easily quantified. Furthermore, the performance of the model cannot be evaluated on a per-instance level in our case, because it shares aspects with the set cover problem. To overcome these problems, we take a holistic approach and develop a rule mining system that simplifies iterative feedback from domain experts and can easily incorporate the domain-specific evaluation needs. A central part of the system is a novel multi-objective anytime rule mining algorithm. The algorithm is based on the GRASP-PR meta-heuristic but extends it with ideas from several other approaches. We successfully applied the system in the industrial context. In the current article, we focus on the description of the algorithm and the concepts of the system. We provide an implementation of the system for reuse.

A large body of research is currently investigating on the connection between machine learning and game theory. In this work, game theory notions are injected into a preference learning framework. Specifically, a preference learning problem is seen as a two-players zero-sum game. An algorithm is proposed to incrementally include new useful features into the hypothesis. This can be particularly important when dealing with a very large number of potential features like, for instance, in relational learning and rule extraction. A game theoretical analysis is used to demonstrate the convergence of the algorithm. Furthermore, leveraging on the natural analogy between features and rules, the resulting models can be easily interpreted by humans. An extensive set of experiments on classification tasks shows the effectiveness of the proposed method in terms of interpretability and feature selection quality, with accuracy at the state-of-the-art.

Exploiting dependencies between labels is considered to be crucial for multi-label classification. Rules are able to expose label dependencies such as implications, subsumptions or exclusions in a human-comprehensible and interpretable manner. However, the induction of rules with multiple labels in the head is particularly challenging, as the number of label combinations which must be taken into account for each rule grows exponentially with the number of available labels. To overcome this limitation, algorithms for exhaustive rule mining typically use properties such as anti-monotonicity or decomposability in order to prune the search space. In the present paper, we examine whether commonly used multi-label evaluation metrics satisfy these properties and therefore are suited to prune the search space for multi-label heads.

Multi-label classification (MLC) is a supervised learning problem in which, contrary to standard multiclass classification, an instance can be associated with several class labels simultaneously. In this chapter, we advocate a rule-based approach to multi-label classification. Rule learning algorithms are often employed when one is not only interested in accurate predictions, but also requires an interpretable theory that can be understood, analyzed, and qualitatively evaluated by domain experts. Ideally, by revealing patterns and regularities contained in the data, a rule-based theory yields new insights in the application domain. Recently, several authors have started to investigate how rule-based models can be used for modeling multi-label data. Discussing this task in detail, we highlight some of the problems that make rule learning considerably more challenging for MLC than for conventional classification. While mainly focusing on our own previous work, we also provide a short overview of related work in this area.

Background: Predictive modeling is a key component of solutions to many healthcare problems. Among all predictive modeling approaches, machine learning methods often achieve the highest prediction accuracy, but suffer from a long-standing open problem precluding their widespread use in healthcare. Most machine learning models give no explanation for their prediction results, whereas interpretability is essential for a predictive model to be adopted in typical healthcare settings. Methods: This paper presents the first complete method for automatically explaining results for any machine learning predictive model without degrading accuracy. We did a computer coding implementation of the method. Using the electronic medical record data set from the Practice Fusion diabetes classification competition containing patient records from all 50 states in the United States, we demonstrated the method on predicting type 2 diabetes diagnosis within the next year. Results: For the champion machine learning model of the competition, our method explained prediction results for 87.4% of patients who were correctly predicted by the model to have type 2 diabetes diagnosis within the next year. Conclusions: Our demonstration showed the feasibility of automatically explaining results for any machine learning predictive model without degrading accuracy.

As part of a plan launched by the Ministry of Health of Brazil to increase the availability of linear accelerators for radiotherapy treatment for the whole country, for which Varian Medical Systems company has won the bidding, a technical cooperation agreement was signed inviting Brazilian Scientific and Technological Institutions to participate in a technology transfer program. As a result, jointly, the Eldorado Research Institute and the Center for Biomedical Engineering of the University of Campinas presents in this work, the concepts behind of a proposed rule engine to aid in the evaluation and decision-making in radiotherapy treatment planning. Normally, the determination of the radiation dose for a given patient is a complex and intensive procedure, which requires a lot of domain knowledge and subjective experience from the oncologists' team. In order to help them in this complex task, and additionally, provide an auxiliary tool for less experienced oncologists, it is presented a project conception of a software system that will make use of a hybrid data-oriented approach. The proposed rule engine will apply both inference mechanism and expression evaluation to verify and accredit the quality of an external beam radiation treatment plan by considering, at first, the 3D-conformal radiotherapy (3DCRT) technique.