Performing statistical inference in high-dimension is an outstanding challenge. A major source of difficulty is the absence of precise information on the distribution of high-dimensional estimators. Here, we consider linear regression in the high-dimensional regime $p\gg n$. In this context, we would like to perform inference on a high-dimensional parameters vector $\theta^*\in{\mathbb R}^p$. Important progress has been achieved in computing confidence intervals for single coordinates $\theta^*_i$. A key role in these new methods is played by a certain debiased estimator $\hat{\theta}^{\rm d}$ that is constructed from the Lasso. Earlier work establishes that, under suitable assumptions on the design matrix, the coordinates of $\hat{\theta}^{\rm d}$ are asymptotically Gaussian provided $\theta^*$ is $s_0$-sparse with $s_0 = o(\sqrt{n}/\log p )$. The condition $s_0 = o(\sqrt{n}/ \log p )$ is stronger than the one for consistent estimation, namely $s_0 = o(n/ \log p)$. We study Gaussian designs with known or unknown population covariance. When the covariance is known, we prove that the debiased estimator is asymptotically Gaussian under the nearly optimal condition $s_0 = o(n/ (\log p)^2)$. Note that earlier work was limited to $s_0 = o(\sqrt{n}/\log p)$ even for perfectly known covariance. The same conclusion holds if the population covariance is unknown but can be estimated sufficiently well, e.g. under the same sparsity conditions on the inverse covariance as assumed by earlier work. For intermediate regimes, we describe the trade-off between sparsity in the coefficients and in the inverse covariance of the design. We further discuss several applications of our results to high-dimensional inference. In particular, we propose a new estimator that is minimax optimal up to a factor $1+o_n(1)$ for i.i.d. Gaussian designs.

We present a first procedure that can estimate -- with statistical consistency guarantees -- any local-maxima of a density, under benign distributional conditions. The procedure estimates all such local maxima, or $\textit{modal-sets}$, of any bounded shape or dimension, including usual point-modes. In practice, modal-sets can arise as dense low-dimensional structures in noisy data, and more generally serve to better model the rich variety of locally-high-density structures in data. The procedure is then shown to be competitive on clustering applications, and moreover is quite stable to a wide range of settings of its tuning parameter.

Human interactions are characterized by explicit as well as implicit channels of communication. While the explicit channel transmits overt messages, the implicit ones transmit hidden messages about the communicator (e.g., his/her intentions and attitudes). There is a growing consensus that providing a computer with the ability to manipulate implicit affective cues should allow for a more meaningful and natural way of studying particular non-verbal signals of human-human communications by human-computer interactions. In this pilot study, we created a non-dynamic human-computer interaction while manipulating three specific non-verbal channels of communication: gaze pattern, facial expression, and gesture. Participants rated the virtual agent on affective dimensional scales (pleasure, arousal, and dominance) while their physiological signal (electrodermal activity, EDA) was captured during the interaction. Assessment of the behavioral data revealed a significant and complex three-way interaction between gaze, gesture, and facial configuration on the dimension of pleasure, as well as a main effect of gesture on the dimension of dominance. These results suggest a complex relationship between different non-verbal cues and the social context in which they are interpreted. Qualifying considerations as well as possible next steps are further discussed in light of these exploratory findings.

The results are a little different to those obtained using RMSE as the objective function. The focus is still well and truly on the volatility indicators, but in this case the best cross validated performance occurred when selecting only 2 out of the 15 candidate variables. Here's a plot of the cross validated performance of the best feature set for various numbers of features: The model clearly performs better in terms of absolute return for a smaller number of predictors. Performance bottoms at 8 predictors and then improves, but never again achieves the performance obtained with 2-4 predictors. This is consistent with Aronson's assertion that we should stick with at most 3-4 variables otherwise overfitting is almost unavoidable.

The above question seems to haunt most people who have been doing statistical predictive modeling before the term machine learning came into play. Nowadays it seems whoever have run a classification problem with any of the advanced algorithms like Neural Network, Support Vector Machine, etc. calls themselves a machine learning expert. But is this machine learning? We have this statistical/mathematical models from the early 60's . And the only reason not all of them was popular back then is because it was too advanced for the computing power available at those times.

The concept of statistical learning started from the method of least squares in the early 1900s has led to the invention of linear regression method. Most of the concepts at those times were applied to astronomical science. The evolution of linear and multiple regression methods gave rise to quantitative statistical computing. Statistical computing divides the majority of the conundrums into two categories. Those are supervised and unsupervised learning categories.

Most of us have looked in the mirror and wondered how good we look. But, it is often difficult to be objective while judging our own attractiveness, and we are often too embarrassed to ask for others' opinion. What if there was a computer program that could answer this question for you, without a human to look at your image? In this post, I will show you how we can use computer vision and machine learning to predict facial attractiveness of an individual from a single photograph. I will use OpenCV, Numpy and scikit-learn to develop a completely automated pipeline that takes a photograph of a person's face, and rates the photo on a scale of 1 to 5. The code is available at the bottom of this page.

Machine learning has become a central part of our life – as consumers, customers, and hopefully as researchers and practitioners! Whether we are applying predictive modeling techniques to our research or business problems, I believe we have one thing in common: We want to make "good" predictions! Fitting a model to our training data is one thing, but how do we know that it generalizes well to unseen data? How do we know that it doesn't simply memorize the data we fed it and fails to make good predictions on future samples, samples that it hasn't seen before? And how do we select a good model in the first place?

Further, recent psychoacoustical evidence suggest the approach bears some resemblance with human auditory processing for sound textures (McDermott et al., 2013; Nelken and de Cheveigné, 2013). In an influential 2007 article, Aucouturier, Defreville & Pachet (Aucouturier et al., 2007) applied a BOF model to categorize both polyphonic music and soundscapes. Their results showed that, while BOF was a meriting model for their polyphonic music dataset, it was spectacularly effective for soundscapes, reaching accuracies of 96%. The contrast, they interpreted, lied in differences in the temporal structure of both types of stimuli, with music being more formally organized and soundscapes more easily summarized by statistics. In a later companion study (Aucouturier and Defreville, 2009), they showed that soundscapes could be time-shuffled without altering listeners' perception of their acoustic similarity, while music could not. While more work was needed for music, the authors therefore concluded that BOF was a sufficient model to approximate human perception for soundscapes, practically ruling out the need to recognize the local acoustic events in a texture in order to identify it.

Heterogeneity is often natural in many contemporary applications involving massive data. While posing new challenges to effective learning, it can play a crucial role in powering meaningful scientific discoveries through the understanding of important differences among subpopulations of interest. In this paper, we exploit multiple networks with Gaussian graphs to encode the connectivity patterns of a large number of features on the subpopulations. To uncover the heterogeneity of these structures across subpopulations, we suggest a new framework of tuning-free heterogeneity pursuit (THP) via large-scale inference, where the number of networks is allowed to diverge. In particular, two new tests, the chi-based test and the linear functional-based test, are introduced and their asymptotic null distributions are established. Under mild regularity conditions, we establish that both tests are optimal in achieving the testable region boundary and the sample size requirement for the latter test is minimal. Both theoretical guarantees and the tuning-free feature stem from efficient multiple-network estimation by our newly suggested approach of heterogeneous group square-root Lasso (HGSL) for high-dimensional multi-response regression with heterogeneous noises. To solve this convex program, we further introduce a tuning-free algorithm that is scalable and enjoys provable convergence to the global optimum. Both computational and theoretical advantages of our procedure are elucidated through simulation and real data examples.