In this video I explain how SVM (Support Vector Machine) algorithm works to classify a linearly separable binary data set. The original presentation is available at http://prezi.com/jdtqiauncqww/?utm_ca...

The first step is often the hardest to take, and when given too much choice in terms of direction it can often be debilitating. This post aims to take a newcomer from minimal knowledge of machine learning in Python all the way to knowledgeable practitioner in 7 steps, all while using freely available materials and resources along the way. The prime objective of this outline is to help you wade through the numerous free options that are available; there are many, to be sure, but which are the best? What is the best order in which to use selected resources? It would probably be helpful to have some basic understanding of one or both of the first 2 topics, but even that won't be necessary; some extra time spent on the earlier steps should help compensate.

We consider the problem of sampling from a strongly log-concave density in $\mathbb{R}^d$, and prove a non-asymptotic upper bound on the mixing time of the Metropolis-adjusted Langevin algorithm (MALA). The method draws samples by running a Markov chain obtained from the discretization of an appropriate Langevin diffusion, combined with an accept-reject step to ensure the correct stationary distribution. Relative to known guarantees for the unadjusted Langevin algorithm (ULA), our bounds show that the use of an accept-reject step in MALA leads to an exponentially improved dependence on the error-tolerance. Concretely, in order to obtain samples with TV error at most $\delta$ for a density with condition number $\kappa$, we show that MALA requires $\mathcal{O} \big(\kappa d \log(1/\delta) \big)$ steps, as compared to the $\mathcal{O} \big(\kappa^2 d/\delta^2 \big)$ steps established in past work on ULA. We also demonstrate the gains of MALA over ULA for weakly log-concave densities. Furthermore, we derive mixing time bounds for a zeroth-order method Metropolized random walk (MRW) and show that it mixes $\mathcal{O}(\kappa d)$ slower than MALA. We provide numerical examples that support our theoretical findings, and demonstrate the potential gains of Metropolis-Hastings adjustment for Langevin-type algorithms.

Learning with streaming data has attracted much attention during the past few years. Though most studies consider data stream with fixed features, in real practice the features may be evolvable. For example, features of data gathered by limited-lifespan sensors will change when these sensors are substituted by new ones. In this paper, we propose a novel learning paradigm: \emph{Feature Evolvable Streaming Learning} where old features would vanish and new features would occur. Rather than relying on only the current features, we attempt to recover the vanished features and exploit it to improve performance. Specifically, we learn two models from the recovered features and the current features, respectively. To benefit from the recovered features, we develop two ensemble methods. In the first method, we combine the predictions from two models and theoretically show that with the assistance of old features, the performance on new features can be improved. In the second approach, we dynamically select the best single prediction and establish a better performance guarantee when the best model switches. Experiments on both synthetic and real data validate the effectiveness of our proposal.

Regression models for supervised learning problems with a continuous target are commonly understood as models for the conditional mean of the target given predictors. This notion is simple and therefore appealing for interpretation and visualisation. Information about the whole underlying conditional distribution is, however, not available from these models. A more general understanding of regression models as models for conditional distributions allows much broader inference from such models, for example the computation of prediction intervals. Several random forest-type algorithms aim at estimating conditional distributions, most prominently quantile regression forests (Meinshausen, 2006, JMLR). We propose a novel approach based on a parametric family of distributions characterised by their transformation function. A dedicated novel "transformation tree" algorithm able to detect distributional changes is developed. Based on these transformation trees, we introduce "transformation forests" as an adaptive local likelihood estimator of conditional distribution functions. The resulting models are fully parametric yet very general and allow broad inference procedures, such as the model-based bootstrap, to be applied in a straightforward way.

In data-mining applications, we are frequently faced with a large fraction of missing entries in the data matrix, which is problematic for most discriminant machine learning algorithms. A solution that we explore in this paper is the use of a generative model (a mixture of Gaussians) to compute the conditional expectation of the missing variables given the observed variables. Since training a Gaussian mixture with many different patterns of missing values can be computationally very expensive, we introduce a spanning-tree based algorithm that significantly speeds up training in these conditions. We also observe that good results can be obtained by using the generative model to fill-in the missing values for a separate discriminant learning algorithm.

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, MONTH 2016 1 Polar n -Complex and n -Bicomplex Singular V alue Decomposition and Principal Component Pursuit Tak-Shing T. Chan, Member, IEEE and Yi-Hsuan Y ang, Member, IEEE Abstract--Informed by recent work on tensor singular value decomposition and circulant algebra matrices, this paper presents a new theoretical bridge that unifies the hypercomplex and tensor-based approaches to singular value decomposition and robust principal component analysis. We begin our work by extending the principal component pursuit to Olariu's polar n - complex numbers as well as their bicomplex counterparts. In so doing, we have derived the polar n -complex and n -bicomplex proximity operators for both the 1-and trace-norm regularizers, which can be used by proximal optimization methods such as the alternating direction method of multipliers. Experimental results on two sets of audio data show that our algebraically-informed formulation outperforms tensor robust principal component analysis. We conclude with the message that an informed definition of the trace norm can bridge the gap between the hypercomplex and tensor-based approaches. Our approach can be seen as a general methodology for generating other principal component pursuit algorithms with proper algebraic structures. I NTRODUCTION T HE robust principal component analysis (RPCA) [1] has received a lot of attention lately in many application areas of signal processing [2]-[5]. Owing to the NPhardness of the above formulation, the principal component pursuit (PCP) [1] has been proposed to solve this relaxed problem instead [6]: min L, S ‖L ‖ λ‖S ‖ 1 s.t. X L S, (2) where ‖·‖ is the trace norm (sum of the singular values),‖·‖ 1 is the entrywise 1-norm, andλ can be set toc/ max(l,m) where c is a positive parameter [1], [2]. The trace norm and the 1-norm are the tightest convex relaxations of the rank and Manuscript received August 26, 2015; revised May 26, 2016 and July 16, 2016; accepted September 3, 2016. This work was supported by a grant from the Ministry of Science and Technology under the contract MOST102-2221-E-001-004-MY3 and the Academia Sinica Career Development Program. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Masahiro Y ukawa. The authors are with the Research Center for Information Technology Innovation, Academia Sinica, Taipei 11564, Taiwan (email: taksh-ingchan@citi.sinica.edu.tw;

In contrast, the rate of convergence for the gradients, that is, the number of iterations T needed to find a point x with ‖ f(x)‖ ε, is "addressed very rarely" and sometimes requires new algorithmic ideas [18]. In particular, in the full-gradient setting, accelerated gradient descent is only suboptimal for this new goal (at least in the worst case), and additional tricks are needed to get the fastest convergence rate [18]. We review these tricks in Section 1.1. Unfortunately, in the stochastic setting, to the best of our knowledge, tight bounds are not known for finding points with small gradients.

Cluster analysis is used to explore structure in unlabeled data sets in a wide range of applications. An important part of cluster analysis is validating the quality of computationally obtained clusters. A large number of different internal indices have been developed for validation in the offline setting. However, this concept has not been extended to the online setting. A key challenge is to find an efficient incremental formulation of an index that can capture both cohesion and separation of the clusters over potentially infinite data streams. In this paper, we develop two online versions (with and without forgetting factors) of the Xie-Beni and Davies-Bouldin internal validity indices, and analyze their characteristics, using two streaming clustering algorithms (sk-means and online ellipsoidal clustering), and illustrate their use in monitoring evolving clusters in streaming data. We also show that incremental cluster validity indices are capable of sending a distress signal to online monitors when evolving clusters go awry. Our numerical examples indicate that the incremental Xie-Beni index with forgetting factor is superior to the other three indices tested.

This guide is intended to be accessible to anyone. Basic concepts in probability, statistics, programming, linear algebra, and calculus will be discussed, but it isn't necessary to have prior knowledge of them to gain value from this series. Artificial intelligence will shape our future more powerfully than any other innovation this century. Anyone who does not understand it will soon find themselves feeling left behind, waking up in a world full of technology that feels more and more like magic. The rate of acceleration is already astounding.