Performance Analysis
Lift (data mining) - Wikipedia
In data mining and association rule learning, lift is a measure of the performance of a targeting model (association rule) at predicting or classifying cases as having an enhanced response (with respect to the population as a whole), measured against a random choice targeting model. A targeting model is doing a good job if the response within the target is much better than the average for the population as a whole. Lift is simply the ratio of these values: target response divided by average response. For example, suppose a population has an average response rate of 5%, but a certain model (or rule) has identified a segment with a response rate of 20%. Then that segment would have a lift of 4.0 (20%/5%).
Editor's picks: The many applications of Machine Learning in banking
Can robots and data stop banks terror financing? Buying a new printer from ISIS is probably not how many people envision their stationary shopping to proceed. But it was only a month ago that the FBI announced that it had found a senior Islamic State (ISIS) official sent money to an alleged operative based in the US via a global financial network that used fake eBay sales to mask payments. This is a timely reminder about how vulnerable businesses can be to terrorist financing. Gurjeet Singh, co-founder and Executive Chairman of Ayasdi, spoke to bobsguide about the challenges of compliance with anti-money laundering, the characteristics of AI, and how AI is vastly improving false positive rates on suspicious security reports. It's a Catch 22: to get financial credit, you need a credit history; to get a credit history, someone has to give you credit.
Global Weisfeiler-Lehman Graph Kernels
Morris, Christopher, Kersting, Kristian, Mutzel, Petra
Most state-of-the-art graph kernels only take local graph properties into account, i.e., the kernel is computed with regard to properties of the neighborhood of vertices or other small substructures. On the other hand, kernels that do take global graph propertiesinto account may not scale well to large graph databases. Here we propose to start exploring the space between local and global graph kernels, striking the balance between both worlds. Specifically, we introduce a novel graph kernel based on the $k$-dimensional Weisfeiler-Lehman algorithm. Unfortunately, the $k$-dimensional Weisfeiler-Lehman algorithm scales exponentially in $k$. Consequently, we devise a stochastic version of the kernel with provable approximation guarantees using conditional Rademacher averages. On bounded-degree graphs, it can even be computed in constant time. We support our theoretical results with experiments on several graph classification benchmarks, showing that our kernels often outperform the state-of-the-art in terms of classification accuracies.
The generalised random dot product graph
Rubin-Delanchy, Patrick, Priebe, Carey E., Tang, Minh
Because they appear in virtually every facet of the digital world, there is considerable value in being able to make inference and predictions based on networks. In Statistics, such endeavours often start with a probability model, mapping unknown quantities of interest to the data, and, here, one is proposed which strikes a promising balance of generality and interpretability. Our focus is on the simplest case of modelling a graph, that is, a set of nodes and (undirected) edges. To start discussions, we consider first the benefits and drawbacks of a foundational model known as the stochastic blockmodel (Holland et al., 1983). In this model, the nodes of the graph can be grouped into k communities, such that the probability of two nodes forming an edge is dependent only on the two communities involved, and is given by a k k inter-community edge probability matrix B. Under basic exchangeability assumptions (Aldous, 1981; Hoover, 1979), the model can be regarded as providing a piecewise constant, or even histogram (Olhede and Wolfe, 2014), approximation to any random graph model satisfying basic exchangeability assumptions (Aldous, 1981; Hoover, 1979). Its generality yet simple interpretation make it a natural candidate for exploratory data analysis and the model is very popular in practice. However, one obvious issue is its discrete structure, in particular, the'hard' assignment of every node to a single community. We would often prefer to describe node behaviour in a more continuous way. In a seminal paper, Hoff et al. (2002) considered a number of latent position models where, in abstract terms, each node i is mapped to a point X
Evaluating Data Science Projects: A Case Study Critique
I've written two blog posts on evaluation--the broccoli of machine learning. Both types are important not only to data scientists but also to managers and executives, who must evaluate project proposals and results. To managers I would say: It's not necessary to understand the inner workings of a machine learning project, but you should understand whether the right things have been measured and whether the results are suited to the business problem. You need to know whether to believe what data scientists are telling you. To this end, here I'll evaluate a machine learning project report.
Text Compression for Sentiment Analysis via Evolutionary Algorithms
Dufourq, Emmanuel, Bassett, Bruce A.
Can textual data be compressed intelligently without losing accuracy in evaluating sentiment? In this study, we propose a novel evolutionary compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression), which makes use of Parts-of-Speech tags to compress text in a way that sacrifices minimal classification accuracy when used in conjunction with sentiment analysis algorithms. An analysis of PARSEC with eight commercial and non-commercial sentiment analysis algorithms on twelve English sentiment data sets reveals that accurate compression is possible with (0%, 1.3%, 3.3%) loss in sentiment classification accuracy for (20%, 50%, 75%) data compression with PARSEC using LingPipe, the most accurate of the sentiment algorithms. Other sentiment analysis algorithms are more severely affected by compression. We conclude that significant compression of text data is possible for sentiment analysis depending on the accuracy demands of the specific application and the specific sentiment analysis algorithm used.
WWE No Mercy 2017: Predictions, Match Card For 'Monday Night Raw' PPV
It's hard to remember a non-WrestleMania or SummerSlam pay-per-view that had two bigger matches than the ones headlining WWE No Mercy 2017 Sunday night. The card features Brock Lesnar vs. Braun Strowman and John Cena vs. Roman Reigns, both of which are WrestleMania-worthy matches. Below are predictions for every match on the WWE No Mercy card, which features wrestlers from "Monday Night Raw." It's time to put the strap on Strowman. Sure, he's gotten a big push by WWE, but his rise to the top of the card has also been an organic one. During a year in which every three-hour "Monday Night Raw" hasn't exactly been worth watching, Strowman has consistently been the best part of the show, going from a monster heel into maybe the most popular wrestler on the roster.
Practical Machine Learning Coursera
About this course: One of the most common tasks performed by data scientists and data analysts are prediction and machine learning. This course will cover the basic components of building and applying prediction functions with an emphasis on practical applications. The course will provide basic grounding in concepts such as training and tests sets, overfitting, and error rates. The course will also introduce a range of model based and algorithmic machine learning methods including regression, classification trees, Naive Bayes, and random forests. The course will cover the complete process of building prediction functions including data collection, feature creation, algorithms, and evaluation.
Model-Powered Conditional Independence Test
Sen, Rajat, Suresh, Ananda Theertha, Shanmugam, Karthikeyan, Dimakis, Alexandros G., Shakkottai, Sanjay
We consider the problem of non-parametric Conditional Independence testing (CI testing) for continuous random variables. Given i.i.d samples from the joint distribution $f(x,y,z)$ of continuous random vectors $X,Y$ and $Z,$ we determine whether $X \perp Y | Z$. We approach this by converting the conditional independence test into a classification problem. This allows us to harness very powerful classifiers like gradient-boosted trees and deep neural networks. These models can handle complex probability distributions and allow us to perform significantly better compared to the prior state of the art, for high-dimensional CI testing. The main technical challenge in the classification problem is the need for samples from the conditional product distribution $f^{CI}(x,y,z) = f(x|z)f(y|z)f(z)$ -- the joint distribution if and only if $X \perp Y | Z.$ -- when given access only to i.i.d. samples from the true joint distribution $f(x,y,z)$. To tackle this problem we propose a novel nearest neighbor bootstrap procedure and theoretically show that our generated samples are indeed close to $f^{CI}$ in terms of total variational distance. We then develop theoretical results regarding the generalization bounds for classification for our problem, which translate into error bounds for CI testing. We provide a novel analysis of Rademacher type classification bounds in the presence of non-i.i.d near-independent samples. We empirically validate the performance of our algorithm on simulated and real datasets and show performance gains over previous methods.