Classifier evasion consists in finding for a given instance $x$ the nearest instance $x'$ such that the classifier predictions of $x$ and $x'$ are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixed Integer Linear Program solver and finds the optimal evading instance under an expressive set of constraints. Our second algorithm trades off optimality for speed by using symbolic prediction, a novel algorithm for fast finite differences on tree ensembles. On a digit recognition task, we demonstrate that both gradient boosted trees and random forests are extremely susceptible to evasions. Finally, we harden a boosted tree model without loss of predictive accuracy by augmenting the training set of each boosting round with evading instances, a technique we call adversarial boosting.

In many machine learning applications, crowdsourcing has become the primary means for label collection. In this paper, we study the optimal error rate for aggregating labels provided by a set of non-expert workers. Under the classic Dawid-Skene model, we establish matching upper and lower bounds with an exact exponent $mI(\pi)$ in which $m$ is the number of workers and $I(\pi)$ the average Chernoff information that characterizes the workers' collective ability. Such an exact characterization of the error exponent allows us to state a precise sample size requirement $m>\frac{1}{I(\pi)}\log\frac{1}{\epsilon}$ in order to achieve an $\epsilon$ misclassification error. In addition, our results imply the optimality of various EM algorithms for crowdsourcing initialized by consistent estimators.

The past few years have seen machine learning emerge as one of the trendiest topics within the technology sector as it allows computers to find hidden insights from the volumes of data being collected without being explicitly programmed where to look. The resources available to process it have increased dramatically. One of the exciting aspects about machine learning is that its applications are virtually endless and in fact it has been transforming a wide variety of industries in interesting ways. In this post, I would like to share my experience using machine learning algorithms, in particular how we, at Capgemini, have developed the capabilities to successfully integrate machine learning-based technology into a framework to help improve service delivery in the public sector. Many machine learning applications are all around us: Amazon and Netflix online recommendation systems, Spotify and Pandora's personalised playlists, Facebook's automatic face recognition and friends recommendations, Google's personalised searches and adds, Uber's prediction of customer demand and pre-location of cars, to mention just a few.

Numerous functional imaging studies have reported neural activities during the experience of specific emotions or cognitive activities and demonstrated the potentials of functional imaging MRI for the classification of cognitive states or identification of mental disorders. In this paper, we consider learning from fMRI data as a pattern recognition problem and mainly focus on how to accurately and stably identify the relevant features (either voxels or network connections) that participate in a given cognitive task or that are closely related with certain mental disorders. In this paper, we will mainly consider the binary classification problems such as discriminating patients of certain mental discorder from the normal persons or classifying different cognative states, though the proposed idea can also be extended to the case of regression As we know, with the rapid development of data capture and storage technologies, the "curse of dimensionality" becomes a common issue in many fields [1] including the field of pattern recognition and machine learning, where "curse of dimensionality" often refers to an extremely high dimensional feature space. Therefore, feature selection, as a way of dimensional reduction, is critical in many pattern recognition applications such as medical image analysis, computer vision, speech recognition and many more [2]. In this paper, we consider the related challeges in the neuroimaging data based pattern recognition, where besides the "curse of dimensionality", feature selection has another common difficulty, which lies in the small number of training samples, due to varied reasons.

Probabilistic modeling is cyclical: we specify a model, infer its posterior, and evaluate its performance. Evaluation drives the cycle, as we revise our model based on how it performs. This requires a metric. Traditionally, predictive accuracy prevails. Yet, predictive accuracy does not tell the whole story. We propose to evaluate a model through posterior dispersion. The idea is to analyze how each datapoint fares in relation to posterior uncertainty around the hidden structure. We propose a family of posterior dispersion indices (PDI) that capture this idea. A PDI identifies rich patterns of model mismatch in three real data examples: voting preferences, supermarket shopping, and population genetics.

Supplying job schedulers with information on how long the jobs are expected to run enabled the development of the backfilling algorithms, which leverage this information to pack the jobs more efficiently and improve system utilization [1]. These algorithms, however, were designed for parallel systems, in which the jobs require many processors in order to execute, and processor fragmentation (idleness) is a big concern. In those environments the scheduler needs to know the actual runtimes of the jobs (use numeric predictions) to be able to optimize the schedule and improve performance [10]. Our work targets systems in which most jobs are serial, like server farms that are used for software testing. In those environments sophisticated scheduling algorithms are not required, and in order to improve performance it is enough to simply separate the short jobs from the long and assign them to different queues in the system [12]. This separation reduces the likelihood that short jobs will be delayed after long ones, improves the average turnaround times of the jobs and overall system throughput.

The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation. In this article, we consider Gaussian latent variable models where the integration over the latent values is approximated using the Laplace method or expectation propagation (EP). We study the properties of several Bayesian leave-one-out (LOO) cross-validation approximations that in most cases can be computed with a small additional cost after forming the posterior approximation given the full data. Our main objective is to assess the accuracy of the approximative LOO cross-validation estimators. That is, for each method (Laplace and EP) we compare the approximate fast computation with the exact brute force LOO computation. Secondarily, we evaluate the accuracy of the Laplace and EP approximations themselves against a ground truth established through extensive Markov chain Monte Carlo simulation. Our empirical results show that the approach based upon a Gaussian approximation to the LOO marginal distribution (the so-called cavity distribution) gives the most accurate and reliable results among the fast methods.

As a fraud practitioner using data mining techniques to detect fraud, anomalies, outliers or other indicators of potential problems I use a combination of data mining and data matching techniques. The volumes of data in a client assignment can vary from 15 million records of company directors, 60,000 employees, accounts payables data of suppliers 900,000 and invoice transaction 11,million. I'm not a great fan of predictive technologies as the disparate data sets don't seem to fit with the techniques, but I'm open to alternative methodologies. I've recently tested a single fraud profile using "Receiver Operating Characteristic" to evaluate the sensitivity and specificity of the profile. The results fell within the ROC space.

For the sake of brevity, not all the relevant data and code are displayed in this post but can rather be found here. And you can visit the final working web application here. The Rugby World Cup (RWC) is here! With many fans around the world excited to see the action unfold over the next month and a half. If you've never heard of the sport, whatisrugby.com

We propose an algorithm for detecting patterns exhibited by anomalous clusters in high dimensional discrete data. Unlike most anomaly detection (AD) methods, which detect individual anomalies, our proposed method detects groups (clusters) of anomalies; i.e. sets of points which collectively exhibit abnormal patterns. In many applications this can lead to better understanding of the nature of the atypical behavior and to identifying the sources of the anomalies. Moreover, we consider the case where the atypical patterns exhibit on only a small (salient) subset of the very high dimensional feature space. Individual AD techniques and techniques that detect anomalies using all the features typically fail to detect such anomalies, but our method can detect such instances collectively, discover the shared anomalous patterns exhibited by them, and identify the subsets of salient features. In this paper, we focus on detecting anomalous topics in a batch of text documents, developing our algorithm based on topic models. Results of our experiments show that our method can accurately detect anomalous topics and salient features (words) under each such topic in a synthetic data set and two real-world text corpora and achieves better performance compared to both standard group AD and individual AD techniques. All required code to reproduce our experiments is available from https://github.com/hsoleimani/ATD