Mixture modeling is a general technique for making any simple model more expressive through weighted combination. This generality and simplicity in part explains the success of the Expectation Maximization (EM) algorithm, in which updates are easy to derive for a wide class of mixture models. However, the likelihood of a mixture model is non-convex, so EM has no known global convergence guarantees. Recently, method of moments approaches offer global guarantees for some mixture models, but they do not extend easily to the range of mixture models that exist. In this work, we present Polymom, an unifying framework based on method of moments in which estimation procedures are easily derivable, just as in EM. Polymom is applicable when the moments of a single mixture component are polynomials of the parameters. Our key observation is that the moments of the mixture model are a mixture of these polynomials, which allows us to cast estimation as a Generalized Moment Problem. We solve its relaxations using semidefinite optimization, and then extract parameters using ideas from computer algebra. This framework allows us to draw insights and apply tools from convex optimization, computer algebra and the theory of moments to study problems in statistical estimation.

Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of R\'enyi-$\alpha$ divergence for a smooth H\"older class of densities on the $d$-dimensional unit cube $[0, 1]^d$. We also illustrate our theoretical results with a numerical experiment.

We analyze a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the $d$-dimensional unit cube $[0,1]^d$ that lie in a $\beta$-H\"older smoothness class, we prove our estimator converges at the rate $O \left( n^{-\frac{\beta}{\beta + d}} \right)$. Furthermore, we prove the estimator is exponentially concentrated about its mean, whereas most previous related results have proven only expected error bounds on estimators.

-- The paper proposes a novel Kernelized image segmentation scheme for noisy images that utilizes the concept of Smallest Univalue Segment Assimilating Nucleus (SUSAN) and incorporates spatial constrai nts by computing circular colour map induced weights. Fuzzy damping coefficients are obtained for each nucleus or center pixel on the basis of the corresponding weighted SUSAN area values, the weights being equal to the inverse of the number of horizontal and vertical moves required to reach a neighborhood pixel from the center pixel. These weights are used to vary the contributions of the different nuclei in the Kernel based framework. The paper also presents an edge quality metric obtained by fuzzy decisi on based edge candidate selection and final computation of the blurriness of the edges after their selection. The inability of existing algorithms to preserve edge information and structural details in their segmented maps necessitates the computation of t he edge quality factor (EQF) for all the competing algorithms. Qualitative and quantitative analysis have been rendered with respect to state - of - the - art algorithms and for images ridden with varying types of noises. Speckle noise ridden SAR images and Rici an noise ridden Magnetic Resonance Images have also been considered for evaluating the effectiveness of the proposed algorithm in extracting important segmentation information. Image segmentation [1] constitutes an important part of image processing which has various applications in the fields of feature extraction and object recognition. The goal of image segmentation methods is to cluster t he pixels of an image into salient regions and hence these methods mainly involve various clustering techniques [2 - 6].

The goal of this paper is to analyze an intriguing phenomenon recently discovered in deep networks, namely their instability to adversarial perturbations (Szegedy et. al., 2014). We provide a theoretical framework for analyzing the robustness of classifiers to adversarial perturbations, and show fundamental upper bounds on the robustness of classifiers. Specifically, we establish a general upper bound on the robustness of classifiers to adversarial perturbations, and then illustrate the obtained upper bound on the families of linear and quadratic classifiers. In both cases, our upper bound depends on a distinguishability measure that captures the notion of difficulty of the classification task. Our results for both classes imply that in tasks involving small distinguishability, no classifier in the considered set will be robust to adversarial perturbations, even if a good accuracy is achieved. Our theoretical framework moreover suggests that the phenomenon of adversarial instability is due to the low flexibility of classifiers, compared to the difficulty of the classification task (captured by the distinguishability). Moreover, we show the existence of a clear distinction between the robustness of a classifier to random noise and its robustness to adversarial perturbations. Specifically, the former is shown to be larger than the latter by a factor that is proportional to \sqrt{d} (with d being the signal dimension) for linear classifiers. This result gives a theoretical explanation for the discrepancy between the two robustness properties in high dimensional problems, which was empirically observed in the context of neural networks. To the best of our knowledge, our results provide the first theoretical work that addresses the phenomenon of adversarial instability recently observed for deep networks. Our analysis is complemented by experimental results on controlled and real-world data.

This class is self paced. You can begin whenever you like and then follow your own pace. It's a good idea to set goals for yourself to make sure you stick with the course. Take a look at the "Class Summary," "What Should I Know," and "What Will I Learn" sections above. If you want to know more, just enroll in the course and start exploring.

Telstra Network Disruptions challenged Kagglers to predict the severity of service disruptions on their network. Using a dataset of features from their service logs, participants were tasked with predicting if a disruption was a momentary glitch or a total interruption of connectivity. Mario Filho, a self-taught data scientist, took first place in his first "solo win". In this blog, he shares a high-level view of his approach. My background in machine learning is completely "self-taught". It all began in 2012 when I decided to learn Calculus on my own through the videos from a MIT class.

Office work is also set to change. Earlier this week, Blue Prism announced plans to debut on the London Stock Exchange. The company, which grew 35 percent in 2015, develops "software robots" which can perform clerical and administration tasks. "Software robots have been deployed successfully and strategically by large, blue chip organizations that have derived tremendous value from this new solution to the labor market," said Alastair Bathgate, the company's co-founder and CEO, in a press release. However, workers should not be overly worried.

Guest blog post by Dalila Benachenhou, originally posted here. Dalila is Professor at George Washington University. In this article, benchmarks were computed on a specific data set, for Geico Calls Prediction, comparing Random Forests, Neural Networks, SVM, FDA, K Nearest Neighbors, C5.0 (Decision Trees), Logistic Regression, and Cart. Introduction: In the first blog, we decided on the predictors. We knew that different predictive models have different assumptions about their predictors.

In machine learning, the problem of reinforcement learning is concerned with using experience gained through interacting with the world and evaluative feedback to improve a system's ability to make behavioral decisions. This tutorial will introduce the fundamental concepts and vocabulary that underlie this field of study. It will also review recent advances in the theory and practice of reinforcement learning, including developments in fundamental technical areas such as generalization, planning, exploration and empirical methodology.