Performance Analysis
Quantifying the relation between performance and success in soccer
Pappalardo, Luca, Cintia, Paolo
The availability of massive data about sports activities offers nowadays the opportunity to quantify the relation between performance and success. In this study, we analyze more than 6,000 games and 10 million events in six European leagues and investigate this relation in soccer competitions. We discover that a team's position in a competition's final ranking is significantly related to its typical performance, as described by a set of technical features extracted from the soccer data. Moreover we find that, while victory and defeats can be explained by the team's performance during a game, it is difficult to detect draws by using a machine learning approach. We then simulate the outcomes of an entire season of each league only relying on technical data, i.e. excluding the goals scored, exploiting a machine learning model trained on data from past seasons. The simulation produces a team ranking (the PC ranking) which is close to the actual ranking, suggesting that a complex systems' view on soccer has the potential of revealing hidden patterns regarding the relation between performance and success.
Active learning in annotating micro-blogs dealing with e-reputation
Cossu, Jean-Valรจre, Molina-Villegas, Alejandro, Tello-Signoret, Mariana
Elections unleash strong political views on Twitter, but what do people really think about politics? Opinion and trend mining on micro blogs dealing with politics has recently attracted researchers in several fields including Information Retrieval and Machine Learning (ML). Since the performance of ML and Natural Language Processing (NLP) approaches are limited by the amount and quality of data available, one promising alternative for some tasks is the automatic propagation of expert annotations. This paper intends to develop a so-called active learning process for automatically annotating French language tweets that deal with the image (i.e., representation, web reputation) of politicians. Our main focus is on the methodology followed to build an original annotated dataset expressing opinion from two French politicians over time. We therefore review state of the art NLP-based ML algorithms to automatically annotate tweets using a manual initiation step as bootstrap. This paper focuses on key issues about active learning while building a large annotated data set from noise. This will be introduced by human annotators, abundance of data and the label distribution across data and entities. In turn, we show that Twitter characteristics such as the author's name or hashtags can be considered as the bearing point to not only improve automatic systems for Opinion Mining (OM) and Topic Classification but also to reduce noise in human annotations. However, a later thorough analysis shows that reducing noise might induce the loss of crucial information.
Scan $B$-Statistic for Kernel Change-Point Detection
Li, Shuang, Xie, Yao, Dai, Hanjun, Song, Le
Detecting the emergence of an abrupt change-point is a classic problem in statistics and machine learning. Kernel-based nonparametric statistics have been used for this task which enjoy fewer assumptions on the distributions than the parametric approach and can handle high-dimensional data. In this paper we focus on the scenario when the amount of background data is large, and propose two related computationally efficient kernel-based statistics for change-point detection, which are inspired by the recently developed $B$-statistics. A novel theoretical result of the paper is the characterization of the tail probability of these statistics using the change-of-measure technique, which focuses on characterizing the tail of the detection statistics rather than obtaining its asymptotic distribution under the null distribution. Such approximations are crucial to control the false alarm rate, which corresponds to the significance level in offline change-point detection and the average-run-length in online change-point detection. Our approximations are shown to be highly accurate. Thus, they provide a convenient way to find detection thresholds for both offline and online cases without the need to resort to the more expensive simulations or bootstrapping. We show that our methods perform well on both synthetic data and real data.
Improving your statistical inferences Coursera
About this course: This course aims to help you to draw better statistical inferences from empirical research. First, we will discuss how to correctly interpret p-values, effect sizes, confidence intervals, Bayes Factors, and likelihood ratios, and how these statistics answer different questions you might be interested in. Then, you will learn how to design experiments where the false positive rate is controlled, and how to decide upon the sample size for your study, for example in order to achieve high statistical power. Subsequently, you will learn how to interpret evidence in the scientific literature given widespread publication bias, for example by learning about p-curve analysis. Finally, we will talk about how to do philosophy of science, theory construction, and cumulative science, including how to perform replication studies, why and how to pre-register your experiment, and how to share your results following Open Science principles.
Lift (data mining) - Wikipedia
In data mining and association rule learning, lift is a measure of the performance of a targeting model (association rule) at predicting or classifying cases as having an enhanced response (with respect to the population as a whole), measured against a random choice targeting model. A targeting model is doing a good job if the response within the target is much better than the average for the population as a whole. Lift is simply the ratio of these values: target response divided by average response. For example, suppose a population has an average response rate of 5%, but a certain model (or rule) has identified a segment with a response rate of 20%. Then that segment would have a lift of 4.0 (20%/5%).
Editor's picks: The many applications of Machine Learning in banking
Can robots and data stop banks terror financing? Buying a new printer from ISIS is probably not how many people envision their stationary shopping to proceed. But it was only a month ago that the FBI announced that it had found a senior Islamic State (ISIS) official sent money to an alleged operative based in the US via a global financial network that used fake eBay sales to mask payments. This is a timely reminder about how vulnerable businesses can be to terrorist financing. Gurjeet Singh, co-founder and Executive Chairman of Ayasdi, spoke to bobsguide about the challenges of compliance with anti-money laundering, the characteristics of AI, and how AI is vastly improving false positive rates on suspicious security reports. It's a Catch 22: to get financial credit, you need a credit history; to get a credit history, someone has to give you credit.
Global Weisfeiler-Lehman Graph Kernels
Morris, Christopher, Kersting, Kristian, Mutzel, Petra
Most state-of-the-art graph kernels only take local graph properties into account, i.e., the kernel is computed with regard to properties of the neighborhood of vertices or other small substructures. On the other hand, kernels that do take global graph propertiesinto account may not scale well to large graph databases. Here we propose to start exploring the space between local and global graph kernels, striking the balance between both worlds. Specifically, we introduce a novel graph kernel based on the $k$-dimensional Weisfeiler-Lehman algorithm. Unfortunately, the $k$-dimensional Weisfeiler-Lehman algorithm scales exponentially in $k$. Consequently, we devise a stochastic version of the kernel with provable approximation guarantees using conditional Rademacher averages. On bounded-degree graphs, it can even be computed in constant time. We support our theoretical results with experiments on several graph classification benchmarks, showing that our kernels often outperform the state-of-the-art in terms of classification accuracies.
The generalised random dot product graph
Rubin-Delanchy, Patrick, Priebe, Carey E., Tang, Minh
Because they appear in virtually every facet of the digital world, there is considerable value in being able to make inference and predictions based on networks. In Statistics, such endeavours often start with a probability model, mapping unknown quantities of interest to the data, and, here, one is proposed which strikes a promising balance of generality and interpretability. Our focus is on the simplest case of modelling a graph, that is, a set of nodes and (undirected) edges. To start discussions, we consider first the benefits and drawbacks of a foundational model known as the stochastic blockmodel (Holland et al., 1983). In this model, the nodes of the graph can be grouped into k communities, such that the probability of two nodes forming an edge is dependent only on the two communities involved, and is given by a k k inter-community edge probability matrix B. Under basic exchangeability assumptions (Aldous, 1981; Hoover, 1979), the model can be regarded as providing a piecewise constant, or even histogram (Olhede and Wolfe, 2014), approximation to any random graph model satisfying basic exchangeability assumptions (Aldous, 1981; Hoover, 1979). Its generality yet simple interpretation make it a natural candidate for exploratory data analysis and the model is very popular in practice. However, one obvious issue is its discrete structure, in particular, the'hard' assignment of every node to a single community. We would often prefer to describe node behaviour in a more continuous way. In a seminal paper, Hoff et al. (2002) considered a number of latent position models where, in abstract terms, each node i is mapped to a point X
Evaluating Data Science Projects: A Case Study Critique
I've written two blog posts on evaluation--the broccoli of machine learning. Both types are important not only to data scientists but also to managers and executives, who must evaluate project proposals and results. To managers I would say: It's not necessary to understand the inner workings of a machine learning project, but you should understand whether the right things have been measured and whether the results are suited to the business problem. You need to know whether to believe what data scientists are telling you. To this end, here I'll evaluate a machine learning project report.