Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, and Newey (2016) provide a generic double/de-biased machine learning (DML) approach for obtaining valid inferential statements about focal parameters, using Neyman-orthogonal scores and cross-fitting, in settings where nuisance parameters are estimated using a new generation of nonparametric fitting methods for high-dimensional data, called machine learning methods. In this note, we illustrate the application of this method in the context of estimating average treatment effects (ATE) and average treatment effects on the treated (ATTE) using observational data. A more general discussion and references to the existing literature are available in Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, and Newey (2016).

This article on going deeper into regression analysis with assumptions, plots & solutions, was posted by Manish Saraswat. Manish who works in marketing and Data Science at Analytics Vidhya believes that education can change this world. R, Data Science and Machine Learning keep him busy. Regression analysis marks the first step in predictive modeling. No doubt, it's fairly easy to implement.

You can solve that optimisation problem using both gradient descent and as a closed-form solution, although for really big datasets it is preferable to use the former since calculating the inverse would be really computationally costly.

This article was posted by Arpan Gupta (Indian Institute of Technology). Let's learn from a precise demo on Fitting Logistic Regression on Titanic Data Set for Machine Learning Description:On April 15, 1912, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew. This tragedy has led to better safety regulations for ships.

There was a verbal solution given to this problem in the members only section. I'm not sure if its "legal" to share the whole thing, but here is an excerpt of the solution. " The time series has a weekly periodicity with two peaks: Monday and Thursday, corresponding respectively to the publication of the Monday and Thursday digests. The impact of the Monday and Thursday email blasts extent over the next day; this makes measuring the yield more difficult, unless you use additional data, e.g. from our newsletter vendor. However, the bulk of the impact is really on Monday and Thursday."

Let's start with a simple problem, we suppose that we have a small dataset with house prices for a specific area in a city, the database contains two fields, the size of the house and its price (SIZE, PRICE), and I would like to know the price of a house with a specific size, the problem is that I don't have that size in my dataset, what should I do? We already know from the title that the solution is linear regression, but to explain more easier, I've a collected a little dataset that contains house prices, in the table below a snippet from the dataset: Visualization helps us a lot in identifying patterns in data, that's way to have a better view to our dataset, I m going to plot it using matplotlib python library: From the plotting we can see that the price grows with the size, but the points don't make a prefect line that can help us predict the price of a new size, so we need to find a linear function h(x) that passes next to all the points but not necessary over them, we call the function the hypothesis: In the equation 2, m is the size of our dataset, Xi is the ith price and Yi is the ith size in the dataset, we call J the error function (or the objective function) that we need to minimize. There are other error functions or estimators in statistics that we can use, but in our case we'll use the MSE or the mean squared error estimator, because it will help us find our unknowns parameters more easier, our function will become: The estimator J takes two arguments, which means it's a 3D function, the figure 3 shows how the function looks like in a 3D graph, our goal here is to find the minimum value, which is the lowest point in the graph below, imagine putting a ball inside the graph, the ball will slide into the bottom of the shape. To find the lowest point in the shape, or in another word minimizing the objective function, we'll use the gradient descent algorithm, which is very simple to understand. To reach the bottom of the shape, we will choose randomly a point in the graph, that's mean setting θ0 and θ1 to a random value, at that point we need to decide, do we need to go up or down?

In coming years residential consumers will face real-time electricity tariffs with energy prices varying day to day, and effective energy saving will require automation - a recommender system, which learns consumer's preferences from her actions. A consumer chooses a scenario of home appliance use to balance her comfort level and the energy bill. We propose a Bayesian learning algorithm to estimate the comfort level function from the history of appliance use. In numeric experiments with datasets generated from a simulation model of a consumer interacting with small home appliances the algorithm outperforms popular regression analysis tools. Our approach can be extended to control an air heating and conditioning system, which is responsible for up to half of a household's energy bill.

Prediction and modelling of competitive sports outcomes has received much recent attention, especially from the Bayesian statistics and machine learning communities. In the real world setting of outcome prediction, the seminal \'{E}l\H{o} update still remains, after more than 50 years, a valuable baseline which is difficult to improve upon, though in its original form it is a heuristic and not a proper statistical "model". Mathematically, the \'{E}l\H{o} rating system is very closely related to the Bradley-Terry models, which are usually used in an explanatory fashion rather than in a predictive supervised or on-line learning setting. Exploiting this close link between these two model classes and some newly observed similarities, we propose a new supervised learning framework with close similarities to logistic regression, low-rank matrix completion and neural networks. Building on it, we formulate a class of structured log-odds models, unifying the desirable properties found in the above: supervised probabilistic prediction of scores and wins/draws/losses, batch/epoch and on-line learning, as well as the possibility to incorporate features in the prediction, without having to sacrifice simplicity, parsimony of the Bradley-Terry models, or computational efficiency of \'{E}l\H{o}'s original approach. We validate the structured log-odds modelling approach in synthetic experiments and English Premier League outcomes, where the added expressivity yields the best predictions reported in the state-of-art, close to the quality of contemporary betting odds.

Subset selection in multiple linear regression is to choose a subset of candidate explanatory variables that tradeoff error and the number of variables selected. We built mathematical programming models for subset selection and compare the performance of an LP-based branch-and-bound algorithm with tailored valid inequalities to known heuristics. We found that our models quickly find a quality solution while the rest of the time is spent to prove optimality. Our models are also applicable with slight modifications to the case with more candidate explanatory variables than observations. For this case, we provide mathematical programming models, propose new criteria, and develop heuristic algorithms based on mathematical programming.

We consider the problem of segmenting a large population of customers into non-overlapping groups with similar preferences, using diverse preference observations such as purchases, ratings, clicks, etc. over subsets of items. We focus on the setting where the universe of items is large (ranging from thousands to millions) and unstructured (lacking well-defined attributes) and each customer provides observations for only a few items. These data characteristics limit the applicability of existing techniques in marketing and machine learning. To overcome these limitations, we propose a model-based projection technique, which transforms the diverse set of observations into a more comparable scale and deals with missing data by projecting the transformed data onto a low-dimensional space. We then cluster the projected data to obtain the customer segments. Theoretically, we derive precise necessary and sufficient conditions that guarantee asymptotic recovery of the true customer segments. Empirically, we demonstrate the speed and performance of our method in two real-world case studies: (a) 84% improvement in the accuracy of new movie recommendations on the MovieLens data set and (b) 6% improvement in the performance of similar item recommendations algorithm on an offline dataset at eBay. We show that our method outperforms standard latent-class and demographic-based techniques.