In high dimensional settings, sparse structures are crucial for efficiency, both in term of memory, computation and performance. It is customary to consider $\ell_1$ penalty to enforce sparsity in such scenarios. Sparsity enforcing methods, the Lasso being a canonical example, are popular candidates to address high dimension. For efficiency, they rely on tuning a parameter trading data fitting versus sparsity. For the Lasso theory to hold this tuning parameter should be proportional to the noise level, yet the latter is often unknown in practice. A possible remedy is to jointly optimize over the regression parameter as well as over the noise level. This has been considered under several names in the literature: Scaled-Lasso, Square-root Lasso, Concomitant Lasso estimation for instance, and could be of interest for confidence sets or uncertainty quantification. In this work, after illustrating numerical difficulties for the Smoothed Concomitant Lasso formulation, we propose a modification we coined Smoothed Concomitant Lasso, aimed at increasing numerical stability. We propose an efficient and accurate solver leading to a computational cost no more expansive than the one for the Lasso. We leverage on standard ingredients behind the success of fast Lasso solvers: a coordinate descent algorithm, combined with safe screening rules to achieve speed efficiency, by eliminating early irrelevant features.

Many predictive and machine learning models have structural or tuning parameters that cannot be directly estimated from the data. For example, when using K-nearest neighbor model, there is no analytical estimator for K (the number of neighbors). Typically, resampling is used to get good performance estimates of the model for a given set of values for K and the one associated with the best results is used. This is basically a grid search procedure. However, there are other approaches that can be used.

I am a chemical engineer who is learning programming. Until now I've mostly been working on building interfaces using web programming. Recently I got access to all of my company's lab/quality, inventory, and PLC/manufacturing data. I am interested in digging into this data. I am (sort of) versed in Python, and have done a few tutorials with sklearn and the linear regression algorithm.

Life scientists often struggle to normalize non-parametric data or ignore normalization prior to data analysis. Based on statistical principles, logarithmic, square-root and arcsine transformations are commonly adopted to normalize non-parametric data for parametric tests. Several other transformations are also available for normalizing data. However, for many, identification of right transformation for non-parametric data is a tricky job. The objective of this paper is to develop a SAS program that identifies right transformation and normalize non-parametric data for regression analysis.

When you go to the grocery store, you see that items of a similar nature are displayed nearby to each other. When you organize the clothes in your closet, you put similar items together (e.g. Every personal organizing tip on the web to save you from your clutter suggests some sort of grouping of similar items together. Even we don't notice it, we are involved in grouping similar objects together in every aspect of our life. This is called clustering in machine learning, so in this post I will provide an overview of data mining clustering methods. In machine learning or data mining, clustering assigns similar objects together in order to discover structures in data that doesn't have any labels.

This project aims at a minimal benchmark for scalability, speed and accuracy of commonly used implementations of a few machine learning algorithms. The target of this study is binary classification with numeric and categorical inputs (of limited cardinality i.e. not very sparse) and no missing data, perhaps the most common problem in business applications (e.g. If the input matrix is of n x p, n is varied as 10K, 100K, 1M, 10M, while p is 1K (after expanding the categoricals into dummy variables/one-hot encoding). This particular type of data structure/size (the largest) stems from this author's interest in some particular business applications. If your software tool of choice is not here, your can benchmark it with minimal work with the following instructions.)

Traditional defenses like firewalls, intrusion detection systems, content filtering proxies and antivirus software have proven to perform successfully in what they were designed to do; however, they were never meant to stop threats that are highly stealthy and polymorphic, like malware is nowadays.

In part 2 we are going focus on checking our assumptions. So far we have learned how to perform a K Means Cluster. When running a K Means Cluster, you first have to choose how many clusters you want. But what is the optimal number of clusters? This is the "art" part of an algorithm like this.

The most common question asked by prospective data scientists is – "What is the best programming language for Machine Learning?" The answer to this question always results in a debate whether to choose R, Python or MATLAB for Machine Learning. Nobody can, in reality, answer the question as to whether Python or R is best language for Machine Learning. However, the programming language one should choose for machine learning directly depends on the requirements of a given data problem, the likes and preferences of the data scientist and the context of machine learning activities they want to perform. According to a survey on Kaggler's Favourite Tools, the open source R programming language turned out to be the favourite among 543 Kagglers of the 1714 Kaggler's listing their data science tools.

The most common question asked by prospective data scientists is – "What is the best programming language for Machine Learning?" The answer to this question always results in a debate whether to choose R, Python or MATLAB for Machine Learning. Nobody can, in reality, answer the question as to whether Python or R is best language for Machine Learning. However, the programming language one should choose for machine learning directly depends on the requirements of a given data problem, the likes and preferences of the data scientist and the context of machine learning activities they want to perform. According to a survey on Kaggler's Favourite Tools, the open source R programming language turned out to be the favourite among 543 Kagglers of the 1714 Kaggler's listing their data science tools.