Many machine learning problems involve Monte Carlo gradient estimators. As a prominent example, we focus on Monte Carlo variational inference (MCVI) in this paper. The performance of MCVI crucially depends on the variance of its stochastic gradients. We propose variance reduction by means of Quasi-Monte Carlo (QMC) sampling. QMC replaces N i.i.d. samples from a uniform probability distribution by a deterministic sequence of samples of length N. This sequence covers the underlying random variable space more evenly than i.i.d. draws, reducing the variance of the gradient estimator. With our novel approach, both the score function and the reparameterization gradient estimators lead to much faster convergence. We also propose a new algorithm for Monte Carlo objectives, where we operate with a constant learning rate and increase the number of QMC samples per iteration. We prove that this way, our algorithm can converge asymptotically at a faster rate than SGD. We furthermore provide theoretical guarantees on QMC for Monte Carlo objectives that go beyond MCVI, and support our findings by several experiments on large-scale data sets from various domains.

Autoencoders are popular among neural-network-based matrix completion models due to their ability to retrieve potential latent factors from the partially observed matrices. Nevertheless, when training data is scarce their performance is significantly degraded due to overfitting. In this paper, we mit- igate overfitting with a data-dependent regularization technique that relies on the principles of multi-task learning. Specifically, we propose an autoencoder-based matrix completion model that performs prediction of the unknown matrix values as a main task, and manifold learning as an auxiliary task. The latter acts as an inductive bias, leading to solutions that generalize better. The proposed model outperforms the existing autoencoder-based models designed for matrix completion, achieving high reconstruction accuracy in well-known datasets.

The folks who brought us the marshmallow test have some unlikely news: children today have more self-control than ever. That conclusion is based on more than 50 years of results from the iconic test, which allows a preschooler to eat one treat immediately or two if she can wait 10 minutes. The effort at delayed gratification is vastly funny but the results were found to have serious implications for children's future success. Led by psychologist Walter Mischel, who created the experiment -- one of the most famous in developmental psychology -- a research team found that children tested between 2002-2012 held out for two minutes longer on average than the original test-takers in the 1960s, and one minute longer than participants in the 1980s. A 4-year-old in the earliest group waited as long as a child between 2 ½ and 3 in the most recent tests, and 4-year-old test-takers in the 1980s waited as long as a child who was 3 ½ in the 2000s.

Scalability may also pose a critical bottleneck one should care about. Each of these meta packages deal with it at different ways.

I am preparing for a workshop on the topic of Artificial Intelligence in Marketing, and what it means for the field. These are some initial thoughts that I penned for this workshop. What is Artificial Intelligence (AI)? The term AI refers to any technological assemblage that can collect inputs from the environment (e.g., through sensors), and take actions as a result of those inputs (e.g., adjust temperature), in ways that simulate human intelligence. This means that the technology can apply rules, can self-correct, and can learn through the acquisition of new information.

Artificial intelligence is an evolving field that requires broad training, so courses typically involve principles of computer science, cognitive psychology and engineering. These are the best artificial intelligence programs. Sign up for Grad Compass to get complete access to U.S. News rankings and school data.

It's great to have you here to talk about data science at GitHub. But before we get there, I want to find out a bit about you, and I want to talk about how you got into data science, what you do at GitHub, but I'd like to take a slightly tangential approach to finding about you first by just asking you what you're thinking about at the moment with respect to data science, or what keeps you up at night, or what really is exciting you? Omoju: The thing I've been thinking about a lot is the term artificial intelligence and the fact that it is such a misnomer because the work that we do is not necessarily artificial intelligence. Most of us in industry don't work on A.I. We work on massive mathematical problems that are basically variants of some kind of linear algebra. And that's what we do. So I've been thinking a lot about that, and then using the right kind of terms, like maybe we're doing things like augmenting human intelligence, or been building like data intensive platforms and ...

Click here to skip the intro and dive right in! When I started the residency program in the summer of 2017, I had a lot of experience programming, and a good understanding of machine learning, but I had never used Tensorflow before. I figured that given my background I'd be able to pick it up quickly. To my surprise, the learning curve was fairly steep, and even months into the residency, I would occasionally find myself confused about how to turn ideas into Tensorflow code. I'm writing this blog post as a message-in-a-bottle to my former self: it's the introduction that I wish I had been given before starting on my journey. Hopefully, it will also be a helpful resource for others. In the three years since its release, Tensorflow has cemented itself as a cornerstone of the deep learning ecosystem.

We introduce a framework based on bilevel programming that unifies gradient-based hyperparameter optimization and meta-learning. We show that an approximate version of the bilevel problem can be solved by taking into explicit account the optimization dynamics for the inner objective. Depending on the specific setting, the outer variables take either the meaning of hyperparameters in a supervised learning problem or parameters of a meta-learner. We provide sufficient conditions under which solutions of the approximate problem converge to those of the exact problem. We instantiate our approach for meta-learning in the case of deep learning where representation layers are treated as hyperparameters shared across a set of training episodes. In experiments, we confirm our theoretical findings, present encouraging results for few-shot learning and contrast the bilevel approach against classical approaches for learning-to-learn.

Ever wondered how an Internet search engine - Google, Yahoo! or Bing - fetches you the correct information in seconds? Or, Amazon fishes out the product you've been looking for in a jiffy? Or, the food delivery app knows so much about restaurants across the country? Well, all these Internet engines run on the same oil: data. More specifically, they look for patterns in a deluge of data to produce the best results for your queries within a fraction of a second.