All rights reserve. 5. Industry Challenge "Predicting ad click–through rates (CTR) is a massive-scale learning problem that is central to the multi-billion dollar online advertising industry" Google Context Ad platforms collect huge data to help them predict ad clicks A good predictive model is essential to serve ads efficiently to optimize over all economic value Sponsored search advertising, contextual advertising, display advertising, and real- time bidding auctions have all relied heavily on the ability of learned models to predict ad click–through rates accurately, quickly, and reliably 5 Private and confidential.

"Words that will Inspire" is an analysis on 2,500 TED talks using text analytics and machine learning on R to find the factors that make some talks more popular than others. What was the motivation for doing this project? I am part of a meetup group called Data Scientist speakers in London that meets regularly to practice data science talks and receive feedback to improve public speaking. Every year at the club we have a competition to see who can come with the best data science story. I joined this competition and wanted to make participate with something special: I wanted to combine my data science skills to analyse famous speeches or talks and use these insights to build an entirely new one.

Supervised learning is the crown jewel of Machine Learning. Supervised learning is the machine learning task or process of producing a function that predicts output variables. It has been adopted widely in the industry. For example, banks apply supervised models to detect credit card fraud. Quantitative traders make purchase decisions based on ML model predictions.

Many scientific questions require estimating the effects of continuous treatments. Outcome modeling and weighted regression based on the generalized propensity score are the most commonly used methods to evaluate continuous effects. However, these techniques may be sensitive to model misspecification, extreme weights or both. In this paper, we propose Kernel Optimal Orthogonality Weighting (KOOW), a convex optimization-based method, for estimating the effects of continuous treatments. KOOW finds weights that minimize the worst-case penalized functional covariance between the continuous treatment and the confounders. This material is based upon work supported by the National Science Foundation under Grants Nos. Using data from the Women's Health Initiative observational study, we apply KOOW to evaluate the effect of red meat consumption on blood pressure. Keywords: Independence, continuous actions, policy evaluation, causal inference, optimization, covariate balance 2 1 Introduction The questions that motivate many scientific studies require estimating the effects of continuous treatments. Continuous treatments are usually indexed by doses and their relationships with the outcome are described by dose-response curves.

We introduce a general approach, called test-time training, for improving the performance of predictive models when test and training data come from different distributions. Test-time training turns a single unlabeled test instance into a self-supervised learning problem, on which we update the model parameters before making a prediction on this instance. We show that this simple idea leads to surprising improvements on diverse image classification benchmarks aimed at evaluating robustness to distribution shifts. Theoretical investigations on a convex model reveal helpful intuitions for when we can expect our approach to help.

Pronounced variability due to the growth of renewable energy sources, flexible loads, and distributed generation is challenging residential distribution systems. This context, motivates well fast, efficient, and robust reactive power control. Real-time optimal reactive power control is possible in theory by solving a non-convex optimization problem based on the exact model of distribution flow. However, lack of high-precision instrumentation and reliable communications, as well as the heavy computational burden of non-convex optimization solvers render computing and implementing the optimal control challenging in practice. Taking a statistical learning viewpoint, the input-output relationship between each grid state and the corresponding optimal reactive power control is parameterized in the present work by a deep neural network, whose unknown weights are learned offline by minimizing the power loss over a number of historical and simulated training pairs. In the inference phase, one just feeds the real-time state vector into the learned neural network to obtain the `optimal' reactive power control with only several matrix-vector multiplications. The merits of this novel statistical learning approach are computational efficiency as well as robustness to random input perturbations. Numerical tests on a 47-bus distribution network using real data corroborate these practical merits.

Training neural-networks is computationally hard. However, in practice they are trained efficiently using gradient-based algorithms, achieving remarkable performance on natural data. To bridge this gap, we observe the property of local correlation: correlation between small patterns of the input and the target label. We focus on learning deep neural-networks with a variant of gradient-descent, when the target function is a tree-structured Boolean circuit. We show that in this case, the existence of correlation between the gates of the circuit and the target label determines whether the optimization succeeds or fails. Using this result, we show that neural-networks can learn the (log n)-parity problem for most product distributions. These results hint that local correlation may play an important role in differentiating between distributions that are hard or easy to learn.

This paper provides a framework to analyze stochastic gradient algorithms in a mean squared error (MSE) sense using the asymptotic normality result of the stochastic gradient descent (SGD) iterates. We perform this analysis by taking the asymptotic normality result and applying it to the finite iteration case. Specifically, we look at problems where the gradient estimators are biased and have reduced variance and compare the iterates generated by these gradient estimators to the iterates generated by the SGD algorithm. We use the work of Fabian to characterize the mean and the variance of the distribution of the iterates in terms of the bias and the covariance matrix of the gradient estimators. We introduce the sliding window SGD (SW-SGD) algorithm, with its proof of convergence, which incurs a lower MSE than the SGD algorithm on quadratic and convex problems. Lastly, we present some numerical results to show the effectiveness of this framework and the superiority of SW-SGD algorithm over the SGD algorithm.

There is no consensus yet on the question whether adaptive gradient methods like Adam are easier to use than non-adaptive optimization methods like SGD. In this work, we fill in the important, yet ambiguous concept of `ease-of-use' by defining an optimizer's \emph{tunability}: How easy is it to find good hyperparameter configurations using automatic random hyperparameter search? We propose a practical and universal quantitative measure for optimizer tunability that can form the basis for a fair optimizer benchmark. Evaluating a variety of optimizers on an extensive set of standard datasets and architectures, we find that Adam is the most tunable for the majority of problems, especially with a low budget for hyperparameter tuning.

Using Pairwise Occurrence Information to Improve Knowledge Graph Completion on Large-Scale Datasets Esma Balkır 1,2*, Masha Naslidnyk 2, Dave Palfrey 2 and Arpit Mittal 2 1 University of Edinburgh, Scotland, UK 2 Amazon Research, Cambridge, UK 1 esma.balkir@ed.ac.uk 2 { naslidny, dpalfrey, mitarpit }@amazon.co.uk Abstract Bilinear models such as DistMult and ComplEx are effective methods for knowledge graph (KG) completion. However, they require large batch sizes, which becomes a performance bottleneck when training on large scale datasets due to memory constraints. In this paper we use occurrences of entity-relation pairs in the dataset to construct a joint learning model and to increase the quality of sampled negatives during training. We show on three standard datasets that when these two techniques are combined, they give a significant improvement in performance, especially when the batch size and the number of generated negative examples are low relative to the size of the dataset. We then apply our techniques to a dataset containing 2 million entities and demonstrate that our model outperforms the baseline by 2.8% absolute on hits@1. 1 Introduction A Knowledge Graph (KG) is a collection of facts which are stored as triples, e.g. Even though knowledge graphs are essential for various NLP tasks, open domain knowledge graphs have missing facts.