We propose a novel probabilistic dimensionality reduction framework that can naturally integrate the generative model and the locality information of data. Based on this framework, we present a new model, which is able to learn a smooth skeleton of embedding points in a low-dimensional space from high-dimensional noisy data. The formulation of the new model can be equivalently interpreted as two coupled learning problem, i.e., structure learning and the learning of projection matrix. This interpretation motivates the learning of the embedding points that can directly form an explicit graph structure. We develop a new method to learn the embedding points that form a spanning tree, which is further extended to obtain a discriminative and compact feature representation for clustering problems. Unlike traditional clustering methods, we assume that centers of clusters should be close to each other if they are connected in a learned graph, and other cluster centers should be distant. This can greatly facilitate data visualization and scientific discovery in downstream analysis. Extensive experiments are performed that demonstrate that the proposed framework is able to obtain discriminative feature representations, and correctly recover the intrinsic structures of various real-world datasets.

Stochastic gradient methods for machine learning and optimization problems are usually analyzed assuming data points are sampled \emph{with} replacement. In practice, however, sampling \emph{without} replacement is very common, easier to implement in many cases, and often performs better. In this paper, we provide competitive convergence guarantees for without-replacement sampling, under various scenarios, for three types of algorithms: Any algorithm with online regret guarantees, stochastic gradient descent, and SVRG. A useful application of our SVRG analysis is a nearly-optimal algorithm for regularized least squares in a distributed setting, in terms of both communication complexity and runtime complexity, when the data is randomly partitioned and the condition number can be as large as the data size per machine (up to logarithmic factors). Our proof techniques combine ideas from stochastic optimization, adversarial online learning, and transductive learning theory, and can potentially be applied to other stochastic optimization and learning problems.

And a more advanced book, by the same authors: The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2009).

It explores main concepts from basic to expert level which can help you achieve better grades, develop your academic career, apply your knowledge at work or make business forecasting related decisions. Learning regression machine learning is indispensable for data mining applications in areas such as consumer analytics, finance, banking, health care, science, e-commerce and social media. It is also essential for academic careers in data mining, applied statistical learning or artificial intelligence. And it is necessary for any business forecasting related decision. But as learning curve can become steep as complexity grows, this course helps by leading you through step by step real world practical examples for greater effectiveness.

Galactic Exchange, Inc. announced the availability of an integrated AppStore embedded within the UI of its flagship ClusterGX product line. California based Galactic Exchange has a vision to hugely simplify the deployment of Hadoop/Spark based infrastructures and associated business intelligence and machine learning applications, both on-premise and in the cloud. The integrated AppStore represents Phase-2 in our singular vision of making the deployment and activation of Big Data infrastructure and applications available to every business," said Rob Mustarde, President and CEO of Galactic Exchange. "Our ClusterGX platform already makes deployment of Hadoop/Spark clusters easier than anything else out there by an order of magnitude. Now with the AppStore, a user can launch enterprise grade applications with a single click." When a user launches an application from the integrated AppStore, ClusterGX will automatically deploy the application across the cluster whilst configuring the vendor specified Hadoop settings. If there are additional dependencies required -- such as Kafka, Impala or others -- ClusterGX will automatically deploy those as well, with no further input required from the user. Rocana is one of the first vendors to partner with Galactic Exchange on the AppStore. The ClusterGX platform represents a step-change in Big Data simplification," said Eric Sammer, CTO and co-founder at Rocana.

The Cooper Hewitt Design Museum houses an impressive collection of designed objects that chronicle the history and significance of design in our evolving world. These objects range from unrealized works of architecture to handwoven textiles from Africa to graphic designed posters that reflect the culture and pulse of humanity of their time. The museum is housed in the former mansion of Andrew Carnegie. Upon its completion in 1901, the sixty-four room mansion was the first private residence in the United States to have a structural steel frame that allowed for more expansive spaces and a feeling of lightness. The Carnegie Mansion was also the first private residence to have a residential elevator, central heating, and a precursor to central AC.

Hierarchical Classification (HC) is a supervised learning problem where unlabeled instances are classified into a taxonomy of classes. Several methods that utilize the hierarchical structure have been developed to improve the HC performance. However, in most cases apriori defined hierarchical structure by domain experts is inconsistent; as a consequence performance improvement is not noticeable in comparison to flat classification methods. We propose a scalable data-driven filter based rewiring approach to modify an expert-defined hierarchy. Experimental comparisons of top-down HC with our modified hierarchy, on a wide range of datasets shows classification performance improvement over the baseline hierarchy (i:e:, defined by expert), clustered hierarchy and flattening based hierarchy modification approaches. In comparison to existing rewiring approaches, our developed method (rewHier) is computationally efficient, enabling it to scale to datasets with large numbers of classes, instances and features. We also show that our modified hierarchy leads to improved classification performance for classes with few training samples in comparison to flat and state-of-the-art HC approaches.

High dimensionality is a frequently confronted problem in many applications of machine learning. It increases time and space requirements for processing the data. Moreover, many machine learning methods tend to over-fit and become less interpretable in the presence of many irrelevant or redundant features. A common way to address this problem is the dimensionality reduction. Principal Component Analysis (PCA) (Jolliffe, 2002) is probably the most widely used dimensionality reduction method. However, it is an unsupervised dimensionality reduction method and does not consider the labels of the data. In order to take the label information into account, supervised dimensionality reduction methods are favored. Linear Discriminant Analysis (LDA) (Anderson, 1968), which is initially proposed as a classification method, is an important supervised dimensionality reduction method.

A new computationally efficient dependence measure, and an adaptive statistical test of independence, are proposed. The dependence measure is the difference between analytic embeddings of the joint distribution and the product of the marginals, evaluated at a finite set of locations (features). These features are chosen so as to maximize a lower bound on the test power, resulting in a test that is data-efficient, and that runs in linear time (with respect to the sample size n). The optimized features can be interpreted as evidence to reject the null hypothesis, indicating regions in the joint domain where the joint distribution and the product of the marginals differ most. Consistency of the independence test is established, for an appropriate choice of features. In real-world benchmarks, independence tests using the optimized features perform comparably to the state-of-the-art quadratic-time HSIC test, and outperform competing O(n) and O(n log n) tests.

We propose a unified and systematic framework for performing online nonnegative matrix factorization in the presence of outliers. Our framework is particularly suited to large-scale data. We propose two solvers based on projected gradient descent and the alternating direction method of multipliers. We prove that the sequence of objective values converges almost surely by appealing to the quasi-martingale convergence theorem. We also show the sequence of learned dictionaries converges to the set of stationary points of the expected loss function almost surely. In addition, we extend our basic problem formulation to various settings with different constraints and regularizers. We also adapt the solvers and analyses to each setting. We perform extensive experiments on both synthetic and real datasets. These experiments demonstrate the computational efficiency and efficacy of our algorithms on tasks such as (parts-based) basis learning, image denoising, shadow removal and foreground-background separation.