In analyzing high-dimensional models, sparsity of the model parameter is a common but often undesirable assumption. In this paper, we study the following two-sample testing problem: given two samples generated by two high-dimensional linear models, we aim to test whether the regression coefficients of the two linear models are identical. We propose a framework named TIERS (short for TestIng Equality of Regression Slopes), which solves the two-sample testing problem without making any assumptions on the sparsity of the regression parameters. TIERS builds a new model by convolving the two samples in such a way that the original hypothesis translates into a new moment condition. A self-normalization construction is then developed to form a moment test. We provide rigorous theory for the developed framework. Under very weak conditions of the feature covariance, we show that the accuracy of the proposed test in controlling Type I errors is robust both to the lack of sparsity in the features and to the heavy tails in the error distribution, even when the sample size is much smaller than the feature dimension. Moreover, we discuss minimax optimality and efficiency properties of the proposed test. Simulation analysis demonstrates excellent finite-sample performance of our test. In deriving the test, we also develop tools that are of independent interest. The test is built upon a novel estimator, called Auto-aDaptive Dantzig Selector (ADDS), which not only automatically chooses an appropriate scale of the error term but also incorporates prior information. To effectively approximate the critical value of the test statistic, we develop a novel high-dimensional plug-in approach that complements the recent advances in Gaussian approximation theory.

Imagine solving mathematical problems where you could use the full physical range of computational possibilities within the laws of the universe, and be inspired by the sublime algorithmic intelligence of the human brain. This is precisely why the emerging field of quantum machine learning (QML) has received so much recent attention. In this blog post, we'd like to discuss the fundamental ideas and applied value of machine learning to computation in general, and then contextualize these ideas in a new way within the paradigm of quantum computation. Machine learning – a subfield of computer science related to computational statistics and pattern recognition – emerged in its modern incarnation in the mid-late 20th century as researchers attempted to build thinking machines. While first-generation artificial intelligence took inspiration from the computers of the 1980s to reason about intelligence and view humans like deterministic, syntactical machines, contemporary artificial intelligence instead chooses to build machines that have the adaptability and variability of human in "coping" with the ill-defined problem of being an individual with incomplete information in a complex world.

This image comes from Xkcd, a webcomic of romance, sarcasm, math, and language. Created by Randall Munroe, he is a CNU graduate with a degree in physics. Before starting xkcd, he worked on robots at NASA's Langley Research Center in Virginia.

We propose a novel kernel based post selection inference (PSI) algorithm, which can not only handle non-linearity in data but also structured output such as multi-dimensional and multi-label outputs. Specifically, we develop a PSI algorithm for independence measures, and propose the Hilbert-Schmidt Independence Criterion (HSIC) based PSI algorithm (hsicInf). The novelty of the proposed algorithm is that it can handle non-linearity and/or structured data through kernels. Namely, the proposed algorithm can be used for wider range of applications including nonlinear multi-class classification and multi-variate regressions, while existing PSI algorithms cannot handle them. Through synthetic experiments, we show that the proposed approach can find a set of statistically significant features for both regression and classification problems. Moreover, we apply the hsicInf algorithm to a real-world data, and show that hsicInf can successfully identify important features.

Before assessing R and Python, I will start with Wolfram Mathematica. You can handle lists and matrices easily, you have all the best mathematical functions, backup of Wolfram Alpha and extremely sophisticated graphics visualizations, that allow you, for instance, to make and visualize an animated gradient descent, animate different weights for a given neural network, choose a specific Machine Learning algorithm and automatically classify your dataset in classes, plot stunning 3D visualizations, make animations and manipulate variables values dynamically at the same time you see the output of your calculation. It has 4.65 Gb size and comes with all libraries integrated. It's a great program when you know the formulae for Machine Learning algorithms, so you can build them from scratch, in a completely customized way. You can also do face recognition, geolocation of objects with 3D plots of map surface, handle cellular automata like any other and develop social networks models with artificial intelligence completely customized.

A couple of weeks ago I wrote an article on simple linear regression, which I would recommend reading before proceeding to read this one. Machine learning is a very interesting topic and I have been studying it on my free time. I hope this article sparks your interest in the subject or helps continue fuel it. In simple linear regression there is a one-to-one relationship between the input variable and the output variable. But in multiple linear regression, as the name implies there is a many-to-one relationship, instead of just using one input variable, you use several.

In this paper, we study how the mean shift algorithm can be used to denoise a dataset. We introduce a new framework to analyze the mean shift algorithm as a denoising approach by viewing the algorithm as an operator on a distribution function. We investigate how the mean shift algorithm changes the distribution and show that data points shifted by the mean shift concentrate around high density regions of the underlying density function. By using the mean shift as a denoising method, we enhance the performance of several clustering techniques, improve the power of two-sample tests, and obtain a new method for anomaly detection.

We propose an algorithm for exploring the entire regularization path of asymmetric-cost linear support vector machines. Empirical evidence suggests the predictive power of support vector machines depends on the regularization parameters of the training algorithms. The algorithms exploring the entire regularization paths have been proposed for single-cost support vector machines thereby providing the complete knowledge on the behavior of the trained model over the hyperparameter space. Considering the problem in two-dimensional hyperparameter space though enables our algorithm to maintain greater flexibility in dealing with special cases and sheds light on problems encountered by algorithms building the paths in one-dimensional spaces. We demonstrate two-dimensional regularization paths for linear support vector machines that we train on synthetic and real data.

The goal of semi-supervised learning is to improve supervised classifiers by using additional unlabeled training examples. In this work we study a simple self-learning approach to semi-supervised learning applied to the least squares classifier. We show that a soft-label and a hard-label variant of self-learning can be derived by applying block coordinate descent to two related but slightly different objective functions. The resulting soft-label approach is related to an idea about dealing with missing data that dates back to the 1930s. We show that the soft-label variant typically outperforms the hard-label variant on benchmark datasets and partially explain this behaviour by studying the relative difficulty of finding good local minima for the corresponding objective functions.

T owards a Theoretical Analysis of PCA for Heteroscedastic Data David Hong, Laura Balzano, and Jeffrey A. Fessler Abstract-- Principal Component Analysis (PCA) is a method for estimating a subspace given noisy samples. It is useful in a variety of problems ranging from dimensionality reduction to anomaly detection and the visualization of high dimensional data. PCA performs well in the presence of moderate noise and even with missing data, but is also sensitive to outliers. PCA is also known to have a phase transition when noise is independent and identically distributed; recovery of the subspace sharply declines at a threshold noise variance. Effective use of PCA requires a rigorous understanding of these behaviors. This paper provides a step towards an analysis of PCA for samples with heteroscedastic noise, that is, samples that have nonuniform noise variances and so are no longer identically distributed. In particular, we provide a simple asymptotic prediction of the recovery of a one-dimensional subspace from noisy heteroscedastic samples. The prediction enables: a) easy and efficient calculation of the asymptotic performance, and b) qualitative reasoning to understand how PCA is impacted by heteroscedasticity (such as outliers).