One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this paper, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization. VI has been used in many applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling. The idea behind VI is to first posit a family of densities and then to find the member of that family which is close to the target. Closeness is measured by Kullback-Leibler divergence. We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive a variant that uses stochastic optimization to scale up to massive data. We discuss modern research in VI and highlight important open problems. VI is powerful, but it is not yet well understood. Our hope in writing this paper is to catalyze statistical research on this class of algorithms.

Large-scale labeled training datasets have enabled deep neural networks to excel on a wide range of benchmark vision tasks. However, in many applications it is prohibitively expensive or time-consuming to obtain large quantities of labeled data. To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled target domain. Unfortunately, direct transfer across domains often performs poorly due to domain shift and dataset bias. Domain adaptation is the machine learning paradigm that aims to learn a model from a source domain that can perform well on a different (but related) target domain. In this paper, we summarize and compare the latest unsupervised domain adaptation methods in computer vision applications. We classify the non-deep approaches into sample re-weighting and intermediate subspace transformation categories, while the deep strategy includes discrepancy-based methods, adversarial generative models, adversarial discriminative models and reconstruction-based methods. We also discuss some potential directions.

Machine Learning (ML) is one of the most exciting and dynamic areas of modern research and application. The purpose of this review is to provide an introduction to the core concepts and tools of machine learning in a manner easily understood and intuitive to physicists. The review begins by covering fundamental concepts in ML and modern statistics such as the bias-variance tradeoff, overfitting, regularization, and generalization before moving on to more advanced topics in both supervised and unsupervised learning. Topics covered in the review include ensemble models, deep learning and neural networks, clustering and data visualization, energy-based models (including MaxEnt models and Restricted Boltzmann Machines), and variational methods. Throughout, we emphasize the many natural connections between ML and statistical physics. A notable aspect of the review is the use of Python notebooks to introduce modern ML/statistical packages to readers using physics-inspired datasets (the Ising Model and Monte-Carlo simulations of supersymmetric decays of proton-proton collisions). We conclude with an extended outlook discussing possible uses of machine learning for furthering our understanding of the physical world as well as open problems in ML where physicists maybe able to contribute. (Notebooks are available at https://physics.bu.edu/~pankajm/MLnotebooks.html )

According to many thought leaders, analysts and early adopters, Artificial Intelligence (AI) and Machine Learning (ML) technologies are rapidly making their way into every industry, geography, system and process. This means that B2B sales and marketers need a primer for artificial intelligence and machine learning to quickly catch up on how they can benefit. This primer on AI and ML will explain how. In general, artificial intelligence is the simulation of human intelligence processes by machines. These processes include learning (the acquisition of information and rules for using the information), reasoning (using the rules to reach approximate or definite conclusions), and self-correction (continuous and tireless learning).

Here we begin our survey of Amazon AWS cloud analytics and big data tools. First we will give an overview of some of what is available. Then we will look at some of them in more detail in subsequent blog posts and provide examples of how to use them. Amazon's approach to selling these cloud services is that these tools take some of the complexity out of developing ML predictive, classification models and neural networks. That is true, but could it be limiting.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

WEBINAR DESCRIPTION This 1 hour session will provide an overview on Multi-layer Artificial Neural Networks (ANNs). Artificial Neural Networks (ANNs) are the building blocks of modern Deep Learning applications, such as image processing, speech recognition, text analytics, driverless cars etc. This session will cover the basics of multi-layer ANNs, discuss forward propagation and backpropagation logic, cost function in a multi-layer ANN and how to achieve convergence. This session will also include demonstration of small python programs which implement such Multi-Layer ANNs. WARNING:- This is an advanced Deep Learning topic.

Machine Learning is about making predictions. This post will give an introduction to Machine Learning through a problem that most businesses face: predicting customer churn. ML can help predict which of your customers are at risk for leaving in advance, and give you an edge by pre-empting with action. Machine Learning can best be understood through four progressive lenses. Machine Learning allows us to accurately predict things using simple statistical methods, algorithms, and modern computing power.

The stochastic gradient descent (SGD) optimization algorithm plays a central role in machine learning and, in particular, deep learning applications such as image analysis and speech recognition (cf., e.g., [12, 13, 16, 23]). It is therefore important to analyze and quantify the convergence speed of the SGD method. There is a vast amount of scientific literature investigating and providing upper bounds for the SGD method and modifications of it (cf., e.g., [3, 4, 5, 6, 7, 8, 9, 10, 11, 18, 20, 21, 24] and cf., e.g., [14] for a more comprehensive review of the literature). Much less attention has been paid to proving lower error bounds for the SGD method, that is, to quantifying the best possible speed of convergence which the SGD method can achieve (cf., e.g., [2, 17, 19, 22, 25]). It is the key contribution of this paper to make a step in this direction.

This morning, MIT Technology Review published its 18th-annual 10 Breakthrough Technologies list, and it includes some fascinating entries. By "breakthrough," the publication means "a technology, or perhaps even a collection of technologies, that will have a profound effect on our lives." Now that we have our definitions out of the way, let's take a look at some of the technologies that our kids will probably see as old-hat, but that we'll consider truly innovative. With the possibility of using metals as well as plastic, composites, and other materials, 3D printing companies will make it easier than ever for businesses to print replacement parts, or even all-new ones. And that means they don't have to keep huge inventories of things they might someday need.