Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of R\'enyi-$\alpha$ divergence for a smooth H\"older class of densities on the $d$-dimensional unit cube $[0, 1]^d$. We also illustrate our theoretical results with a numerical experiment.

The paper proposes a novel Kernelized image segmentation scheme for noisy images that utilizes the concept of Smallest Univalue Segment Assimilating Nucleus (SUSAN) and incorporates spatial constraints by computing circular colour map induced weights. Fuzzy damping coefficients are obtained for each nucleus or center pixel on the basis of the corresponding weighted SUSAN area values, the weights being equal to the inverse of the number of horizontal and vertical moves required to reach a neighborhood pixel from the center pixel. These weights are used to vary the contributions of the different nuclei in the Kernel based framework. The paper also presents an edge quality metric obtained by fuzzy decision based edge candidate selection and final computation of the blurriness of the edges after their selection. The inability of existing algorithms to preserve edge information and structural details in their segmented maps necessitates the computation of the edge quality factor (EQF) for all the competing algorithms. Qualitative and quantitative analysis have been rendered with respect to state-of-the-art algorithms and for images ridden with varying types of noises. Speckle noise ridden SAR images and Rician noise ridden Magnetic Resonance Images have also been considered for evaluating the effectiveness of the proposed algorithm in extracting important segmentation information.

Mixture modeling is a general technique for making any simple model more expressive through weighted combination. This generality and simplicity in part explains the success of the Expectation Maximization (EM) algorithm, in which updates are easy to derive for a wide class of mixture models. However, the likelihood of a mixture model is non-convex, so EM has no known global convergence guarantees. Recently, method of moments approaches offer global guarantees for some mixture models, but they do not extend easily to the range of mixture models that exist. In this work, we present Polymom, an unifying framework based on method of moments in which estimation procedures are easily derivable, just as in EM. Polymom is applicable when the moments of a single mixture component are polynomials of the parameters. Our key observation is that the moments of the mixture model are a mixture of these polynomials, which allows us to cast estimation as a Generalized Moment Problem. We solve its relaxations using semidefinite optimization, and then extract parameters using ideas from computer algebra. This framework allows us to draw insights and apply tools from convex optimization, computer algebra and the theory of moments to study problems in statistical estimation.

Guest blog post by Dalila Benachenhou, originally posted here. Dalila is Professor at George Washington University. In this article, benchmarks were computed on a specific data set, for Geico Calls Prediction, comparing Random Forests, Neural Networks, SVM, FDA, K Nearest Neighbors, C5.0 (Decision Trees), Logistic Regression, and Cart. Introduction: In the first blog, we decided on the predictors. We knew that different predictive models have different assumptions about their predictors.

Machine Learning is nothing but building a'machine' which'learns' from its experience. And, becomes better with experience – just like humans. We also learn from our experiences. Companies like Google, Facebook, Microsoft are using machine learning techniques at a larger scale. However, one common mis-conception people have is that they need to learn coding to start machine learning.

Here we discuss general applications of statistical models, whether they arise from data science, operations research, engineering, machine learning or statistics. We do not discuss specific algorithms such as decision trees, logistic regression, Bayesian modeling, Markov models, data reduction or feature selection. Instead, I discuss frameworks - each one using its own types of techniques and algorithms - to solve real life problems. Most of the entries below are found in Wikipedia, and I have used a few definitions or extracts from the relevant Wikipedia articles, in addition to personal contributions. Spatial dependency is the co-variation of properties within geographic space: characteristics at proximal locations appear to be correlated, either positively or negatively. Methods for time series analyses may be divided into two classes: frequency-domain methods and time-domain methods.

Machine learning explores the study and construction of algorithms that learn and make predictions based on data. In the field of machine learning, data scientists, who specialize in analyzing data, are responsible for writing and modifying such algorithms. Initially, a data scientist writes an algorithm based on a set of data features. This is generally an iterative process in which the data scientist explores different algorithms for predictive purpose. In this process, the amount of data and the number of features chosen for analysis may change.

This is the 2nd part of the series. In this part we'll discuss how to choose between Logistic Regression, Decision Trees and Support Vector Machines. The most correct answer as mentioned in the first part of this 2 part article, still remains it depends. We'll continue our effort to shed some light on, it depends on what. All three of these techniques have certain properties inherent by their design, we'll elaborate on some in order to provide you with few pointers on their selection for your particular business problem.

The createPairwiseMatches() function shown in Figure 7 above, extracts features in parallel mapping images to vertical and horizontal luminosity histograms. Furthermore, the histograms for each image are saved in a hash table for quick reference since each image's features will be repeatedly matched to other images. Once the match features are extracted, the match is immediately placed in a thread safe blocking collection for further downstream reduction processing. While the mapping functions shown in Figure 7 are executing in a background thread, parallel reduce functions simultaneously execute processing each completed match produced to calculate the similarity between the match images.

The nonnegative matrix factorization (NMF) is widely used in signal and image processing, including bio-informatics, blind source separation and hyperspectral image analysis in remote sensing. A great challenge arises when dealing with a nonlinear formulation of the NMF. Within the framework of kernel machines, the models suggested in the literature do not allow the representation of the factorization matrices, which is a fallout of the curse of the pre-image. In this paper, we propose a novel kernel-based model for the NMF that does not suffer from the pre-image problem, by investigating the estimation of the factorization matrices directly in the input space. For different kernel functions, we describe two schemes for iterative algorithms: an additive update rule based on a gradient descent scheme and a multiplicative update rule in the same spirit as in the Lee and Seung algorithm. Within the proposed framework, we develop several extensions to incorporate constraints, including sparseness, smoothness, and spatial regularization with a total-variation-like penalty. The effectiveness of the proposed method is demonstrated with the problem of unmixing hyperspectral images, using well-known real images and results with state-of-the-art techniques.