K-Nearest Neighbors (K-NN) is one of the simplest machine learning algorithms. Like other machine learning techniques, it was inspired by human reasoning. For example, when something significant happens in your life, you memorize that experience and use it as a guideline for future decisions. Let me give you a scenario of a person dropping a glass. While the glass is falling, you've made the prediction that the glass will break when it hits the ground.

Product categories are the structural backbone of every online shop, but it can be quite a nightmare for e-commerce managers to make sure that all products are assigned to the correct categories. The set of available categories is typically large (Amazon has listed over 50000), changes constantly, and new products have to be added on a daily basis. Mistakes can be costly, because miscategorized products not only look confusing and unprofessional, they also cannot be sold when customers are not able to find them. To improve the process of product categorization, we looked into methods from machine learning. Our goal was to develop a machine learning system that can predict which categories fit best to a given product, in order to make the whole process easier, faster and less error-prone. In this blog post, I am going to walk you through the problems we faced on the way and how we decided to solve them.

Time series forecasting, such as predictions of sales values and volumes, can be a challenging problem, particularly when classical statistical methods do not cope well with the complexity of the data. Forecasting with these methods can be time-consuming, taking weeks or months at a time to deliver. Machine Learning (ML) algorithms can detect complex relationships and trends within the data and create accurate sales forecasts nearly in real time. ML is therefore becoming an increasingly valuable tool for brand owners, enhancing their ability to make the right decisions faster. In our previous blog in this series, titled An introduction to using machine learning in marketing – for classification problems, we discussed the basics of machine learning, the Random Forest algorithm for classification problems and how it can be used in Smart Marketing campaigns.

A classifier is a tool in data mining that takes a bunch of data representing things we want to classify and attempts to predict which class the new data belongs to. Orange, an open-source data visualization and analysis tool for data mining, implements C4.5 in their decision tree classifier. Support vector machine (SVM) learns a hyperplane to classify data into 2 classes. The balls represent data points, and the red and blue color represent 2 classes.

Instance- and label-dependent label noise (ILN) is widely existed in real-world datasets but has been rarely studied. In this paper, we focus on a particular case of ILN where the label noise rates, representing the probabilities that the true labels of examples flip into the corrupted labels, have upper bounds. We propose to handle this bounded instance- and label-dependent label noise under two different conditions. First, theoretically, we prove that when the marginal distributions $P(X|Y=+1)$ and $P(X|Y=-1)$ have non-overlapping supports, we can recover every noisy example's true label and perform supervised learning directly on the cleansed examples. Second, for the overlapping situation, we propose a novel approach to learn a well-performing classifier which needs only a few noisy examples to be labeled manually. Experimental results demonstrate that our method works well on both synthetic and real-world datasets.

Brain-Computer Interface (BCI) uses brain signals in order to provide a new method for communication between human and outside world. Feature extraction, selection and classification are among the main matters of concerns in signal processing stage of BCI. In this article, we present our findings about the most effective features and classifiers in some brain tasks. Six different groups of classical features and twelve classifiers have been examined in nine datasets of brain signal. The results indicate that energy of brain signals in {\alpha} and \b{eta} frequency bands, together with some statistical parameters are more effective, comparing to the other types of extracted features. In addition, Bayesian classifier with Gaussian distribution assumption and also Support Vector Machine (SVM) show to classify different BCI datasets more accurately than the other classifiers. We believe that the results can give an insight about a strategy for blind classification of brain signals in brain-computer interface.

The Expectation-Maximization (EM) algorithm is a widely used technique for parameter estimation. It is an iterative procedure that monotonically increases the likelihood. When the likelihood is not concave, it is well known that EM can converge to a non-global optimum. However, recent work has sidestepped the question of whether EM reaches the likelihood maximizer, instead by directly working out statistical guarantees on its loss. These 1 explorations have identified regions of initialization for which the EM estimate approaches the true parameter in probability, assuming the model is well-specified. This line of research was spurred by [1] which established general conditions for which a ball centered at the true parameter would be a basin of attraction for the population version of the EM operator. For a large enough sample size, the difference (in that ball) between the sample EM operator and the population EM operator can be bounded such that the EM estimate approaches the true parameter with high probability. That bound is the sum of two terms with distinct interpretations.

Minimum energy paths for transitions such as atomic and/or spin rearrangements in thermalized systems are the transition paths of largest statistical weight. Such paths are frequently calculated using the nudged elastic band method, where an initial path is iteratively shifted to the nearest minimum energy path. The computational effort can be large, especially when ab initio or electron density functional calculations are used to evaluate the energy and atomic forces. Here, we show how the number of such evaluations can be reduced by an order of magnitude using a Gaussian process regression approach where an approximate energy surface is generated and refined in each iteration. When the goal is to evaluate the transition rate within harmonic transition state theory, the evaluation of the Hessian matrix at the initial and final state minima can be carried out beforehand and used as input in the minimum energy path calculation, thereby improving stability and reducing the number of iterations needed for convergence. A Gaussian process model also provides an uncertainty estimate for the approximate energy surface, and this can be used to focus the calculations on the lesser-known part of the path, thereby reducing the number of needed energy and force evaluations to a half in the present calculations. The methodology is illustrated using the two-dimensional M\"uller-Brown potential surface and performance assessed on an established benchmark involving 13 rearrangement transitions of a heptamer island on a solid surface.

Sampling is a fundamental topic in graph signal processing, having found applications in estimation, clustering, and video compression. In contrast to traditional signal processing, the irregularity of the signal domain makes selecting a sampling set non-trivial and hard to analyze. Indeed, though conditions for graph signal interpolation from noiseless samples exist, they do not lead to a unique sampling set. The presence of noise makes choosing among these sampling sets a hard combinatorial problem. Although greedy sampling schemes are commonly used in practice, they have no performance guarantee. This work takes a twofold approach to address this issue. First, universal performance bounds are derived for the Bayesian estimation of graph signals from noisy samples. In contrast to currently available bounds, they are not restricted to specific sampling schemes and hold for any sampling sets. Second, this paper provides near-optimal guarantees for greedy sampling by introducing the concept of approximate submodularity and updating the classical greedy bound. It then provides explicit bounds on the approximate supermodularity of the interpolation mean-square error showing that it can be optimized with worst-case guarantees using greedy search even though it is not supermodular. Simulations illustrate the derived bound for different graph models and show an application of graph signal sampling to reduce the complexity of kernel principal component analysis.

Recently I came across this idea of center loss described in this paper. You define the outputs from the second last layers of the neural network as embeddings. For this loss, you define a per class center which serves as the centroid of embeddings corresponding to that class. As the network gets updated with gradient descent, the per class center term needs to be updated. Ideally the update would involve going through the entire training data, but that is not feasible in practice.