In this post, we are going to introduce you to the Support Vector Machine (SVM) machine learning algorithm. We will follow a similar process to our recent post Naive Bayes for Dummies; A Simple Explanation by keeping it short and not overly-technical. The aim is to give those of you who are new to machine learning a basic understanding of the key concepts of this algorithm. A Support Vector Machine (SVM) is a supervised machine learning algorithm that can be employed for both classification and regression purposes. SVMs are more commonly used in classification problems and as such, this is what we will focus on in this post.

There is definitely an established machine learning ecosystem, or, perhaps more accurately, a small set of established machine learning ecosystems. For research it would seem that the undisputed champion of machine learning ecosystems is centered on Python and its many libraries which support the data preparation and subsequent machine learning process itself, whether it be via scikit-learn, one of the many deep learning libraries available, or home-spun and highly specialized tools for achieving the same goals. This says nothing of the great support tools that grow up around the edges of the ecosystem, some of which become polished and useful enough to carve out their own eventual niche. As those in industry would be the first to let me know, Python is not the only option. There are Java-based tools (Deeplearning4j, Weka), those integrated with Apache Spark and/or Hadoop (MLlib, Mahout), C solutions (TensorFlow is written in C, as are many others in the Python ecosystem), and even those for Clojure, F#, Rust, and a whole host of other languages, environments, and ecosystems.

Note: Before starting Part 2, be sure to read Part 1! When it comes to machine learning, ultimately the most important picture to have is the big picture. Whether it's logistic regression, random forests, Bayesian methods, support vector machines, or neural nets, everyone seems to have their favorite! Unfortunately these discussions tend to truncate the challenges of machine learning into a single problem, which is a particularly problematic misrepresentation for people who are just getting started with machine learning. Sure, picking a good model is important, but it's certainly not enough (and it's debatable whether a model can actually be'good' devoid of the context of the domain, the hypothesis, the shape of the data, and the intended application. In this post we'll discuss model selection in the context of the big picture, which I'll present in terms of the model selection triple, and we'll explore a set of visual tools for navigating the triple.

By the end of this article, you'll understand how Numerai is using advances in cryptography like homomorphic encryption to allow for open participation in the problem of stock market efficiency. Over the last few years, machine learning algorithms solved big problems in computer vision. One such problem was getting an algorithm to learn how to recognize handwritten digits in the MNIST dataset. Everyone writes digits differently, so the problem was difficult for computers to grasp. When the dataset first became available in 1998, machine learning algorithms for computer vision were not very accurate.

XGBoost (short for Extreme Gradient Boosting) is a relatively new classification technique in machine learning which has won more and more popularity because of its exceptional performance in multiple competitions hosted on Kaggle.com. A lesser known benefit of using XGBoost is that the tree ensemble model can rank features for high-dimensional data sets. The official implementation of XGBoost (Python) provides only one feature scoring function called get_fscore. What it does is that, it computes feature scores by counting how many times a feature appears in the splits and rank the features according to the splits. It is simple, and it is straightforward, but I believe we should not ignore another metric which is critical to the decision tree method.

It has been observed in a variety of contexts that gradient descent methods have great success in solving low-rank matrix factorization problems, despite the relevant problem formulation being non-convex. We tackle a particular instance of this scenario, where we seek the $d$-dimensional subspace spanned by a streaming data matrix. We apply the natural first order incremental gradient descent method, constraining the gradient method to the Grassmannian. In this paper, we propose an adaptive step size scheme that is greedy for the noiseless case, that maximizes the improvement of our metric of convergence at each data index $t$, and yields an expected improvement for the noisy case. We show that, with noise-free data, this method converges from any random initialization to the global minimum of the problem. For noisy data, we provide the expected convergence rate of the proposed algorithm per iteration.

A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a "tuning function" for each neuron. Unfortunately, because this approach handles neurons individually, it cannot take advantage of simultaneous measurements from spatially adjacent neurons that often have similar tuning properties. On the other hand, sharing information between adjacent neurons can errantly degrade estimates of tuning functions across space if there are sharp discontinuities in tuning between nearby neurons. In this paper, we develop a computationally efficient block Gibbs sampler that effectively pools information between neurons to de-noise tuning function estimates while simultaneously preserving sharp discontinuities that might exist in the organization of tuning across space. This method is fully Bayesian and its computational cost per iteration scales sub-quadratically with total parameter dimensionality. We demonstrate the robustness and scalability of this approach by applying it to both real and synthetic datasets. In particular, an application to data from the spinal cord illustrates that the proposed methods can dramatically decrease the experimental time required to accurately estimate tuning functions.

In this paper, we consider the problem of modeling discrete social network data and learning the underlying group dynamics. The goal is to develop probabilistic profiles of large collections of data while preserving the essential temporal relationships that provide insights for various applications of interest. For example, in social network analysis, we want to analyze relationships between social agents and their behaviors over time and on various social media sites (i.e., Facebook, Twitter, Instagram, Google, etc.). In web advertising analysis, we want to analyze the relationships between customers and the types of products they buy from different shopping sites to capture customers' buying behaviors and learn the intrinsic factors that effect their buying decision process. In the study of scientific collaboration, using co-authorship networks from multiple journals on related subjects, one can analyze relationships between subjects and authors.

Pruning of redundant or irrelevant instances of data is a key to every successful solution for pattern recognition. In this paper, we present a novel ranking-selection framework for low-length but highly correlated instances. Instead of working in the low-dimensional instance space, we learn a supervised projection to high-dimensional space spanned by the number of classes in the dataset under study. Imposing higher distinctions via exposing the notion of labels to the instances, lets to deploy one versus all ranking for each individual classes and selecting quality instances via adaptive thresholding of the overall scores. To prove the efficiency of our paradigm, we employ it for the purpose of texture understanding which is a hard recognition challenge due to high similarity of texture pixels and low dimensionality of their color features. Our experiments show considerable improvements in recognition performance over other local descriptors on several publicly available datasets.

The term "data science" was first used by the statistician William H. Cleveland in his 2001 paper entitled, "Data Science: An Action Plan for Expanding the Technical Areas of the field of Statistics". Cleveland emphasized that the "[results in] data science should be judged by the extent to which they enable the analyst to learn from data". The scientific discipline of learning from data has been happening for centuries before the term data science ever came into being. Statisticians have been collecting, processing, analysing, visualising and interpreting vast amounts of diverse data to generate models. In doing so, they developed many algorithms that are used for regression and classification such as GLM (Generalised Linear Modeling) and embedded in statistical packages such as SAS and SPSS that are used extensively to this day.