Fan, Fenglei, Shan, Hongming, Wang, Ge

For manifold learning, it is assumed that high-dimensional sample/data points are on an embedded low-dimensional manifold. Usually, distances among samples are computed to represent the underlying data structure, for a specified distance measure such as the Euclidean distance or geodesic distance. For manifold learning, here we propose a metric according to the angular change along a geodesic line, thereby reflecting the underlying shape-oriented information or the similarity between high- and low-dimensional representations of a data cloud. Our numerical results are described to demonstrate the feasibility and merits of the proposed dimensionality reduction scheme

Data forms the foundation of any machine learning algorithm, without it, Data Science can not happen. Sometimes, it can contain a huge number of features, some of which are not even required. Such redundant information makes modeling complicated. Furthermore, interpreting and understanding the data by visualization gets difficult because of the high dimensionality. This is where dimensionality reduction comes into play. Dimensionality reduction is the task of reducing the number of features in a dataset. In machine learning tasks like regression or classification, there are often too many variables to work with. These variables are also called features.

Why Do We Need to Reduce the Dimensionality? A high-dimensional dataset is a dataset that has a great number of columns (or variables). Such a dataset presents many mathematical or computational challenges. The good news is that variables (or called features) are often correlated. We can find a subset of the variables to represent the same level of information in the data, or transform the variables to a new set of variables without losing much information.

Torresani, Lorenzo, Lee, Kuang-chih

Metric learning has been shown to significantly improve the accuracy of k-nearest neighbor (kNN) classification. In problems involving thousands of features, distance learning algorithms cannot be used due to overfitting and high computational complexity. In such cases, previous work has relied on a two-step solution: first apply dimensionality reduction methods to the data, and then learn a metric in the resulting low-dimensional subspace. In this paper we show that better classification performance can be achieved by unifying the objectives of dimensionality reduction and metric learning. We propose a method that solves for the low-dimensional projection of the inputs, which minimizes a metric objective aimed at separating points in different classes by a large margin. This projection is defined by a significantly smaller number of parameters than metrics learned in input space, and thus our optimization reduces the risks of overfitting. Theory and results are presented for both a linear as well as a kernelized version of the algorithm. Overall, we achieve classification rates similar, and in several cases superior, to those of support vector machines.

Torresani, Lorenzo, Lee, Kuang-chih