Inference networks of traditional Variational Autoencoders (VAEs) are typically amortized, resulting in relatively inaccurate posterior approximation compared to instance-wise variational optimization. Recent semi-amortized approaches were proposed to address this drawback; however, their iterative gradient update procedures can be computationally demanding. To address these issues, in this paper we introduce an accurate amortized inference algorithm. We propose a novel recursive mixture estimation algorithm for VAEs that iteratively augments the current mixture with new components so as to maximally reduce the divergence between the variational and the true posteriors. Using the functional gradient approach, we devise an intuitive learning criteria for selecting a new mixture component: the new component has to improve the data likelihood (lower bound) and, at the same time, be as divergent from the current mixture distribution as possible, thus increasing representational diversity. Compared to recently proposed boosted variational inference (BVI), our method relies on amortized inference in contrast to BVI's non-amortized single optimization instance. A crucial benefit of our approach is that the inference at test time requires a single feed-forward pass through the mixture inference network, making it significantly faster than the semi-amortized approaches. We show that our approach yields higher test data likelihood than the state-of-the-art on several benchmark datasets.

Out-of-distribution generalization is a core challenge in machine learning. We introduce and propose a solution to a new type of out-of-distribution evaluation, which we call close category generalization. This task specifies how a classifier should extrapolate to unseen classes by considering a bi-criteria objective: (i) on in-distribution examples, output the correct label, and (ii) on out-of-distribution examples, output the label of the nearest neighbor in the training set. In addition to formalizing this problem, we present a new training algorithm to improve the close category generalization of neural networks. We compare to many baselines, including robust algorithms and out-of-distribution detection methods, and we show that our method has better or comparable close category generalization. Then, we investigate a related representation learning task, and we find that performing well on close category generalization correlates with learning a good representation of an unseen class and with finding a good initialization for few-shot learning. Code available at https://github.com/yangarbiter/close-category-generalization

In this work, we consider the application of model-based deep learning in nonlinear principal component analysis (PCA). Inspired by the deep unfolding methodology, we propose a task-based deep learning approach, referred to as Deep-RLS, that unfolds the iterations of the well-known recursive least squares (RLS) algorithm into the layers of a deep neural network in order to perform nonlinear PCA. In particular, we formulate the nonlinear PCA for the blind source separation (BSS) problem and show through numerical analysis that Deep-RLS results in a significant improvement in the accuracy of recovering the source signals in BSS when compared to the traditional RLS algorithm.

The future won't be made by either humans or machines alone, but by both, working together. Technologies modeled on how human brains work are already augmenting people's abilities, and will only get more influential as society gets used to these increasingly capable machines. Technology optimists have envisioned a world with rising human productivity and quality of life as Artificial Intelligence systems take over life's drudgery and administrivia, benefiting everyone. Pessimists, on the other hand, have warned that these advances could come at great cost in lost jobs and disrupted lives. And fearmongers worry that AI might eventually make human beings obsolete.

High-quality machine learning (ML) models depend on accurately labeled, high-quality training, validation, and test data. As ML and deep learning models are increasingly integrated into production environments, it's becoming more important than ever to have customizable, real-time data labeling pipelines that can continuously receive and process unlabeled data. For example, you may want to create a consumer-facing application that regularly collects and sends new data objects to a data labeling pipeline, which produces labels and builds a dataset for model training or retraining. This pipeline creates a positive feedback loop that leads to more accurate, sophisticated models. Amazon SageMaker Ground Truth streaming labeling jobs provide infrastructure and resources to create a continuously running labeling job that receives new data objects on demand and sends them to human workers to be labeled. You can chain multiple streaming labeling jobs together to create more intricate and refined data labeling pipelines. Use this blog post to learn how to set up and customize Ground Truth streaming labeling jobs.

While most AI in use today can be classified as early-stage advanced analytics, some enterprises have built large data science teams to apply machine learning and deep learning algorithms to business processes. Many of these enterprises have built the necessary support infrastructure to train these algorithms on large data sets, deploying the resulting AI models to production to generate business insights. However, many consumer and business consumption patterns changed dramatically in 2020, causing these advanced AI models to fail or behave erratically. Many of these models that have been trained to make predictions based on historical data have not been able to deal with the data anomalies created by disruptive business conditions and changing preferences. Companies using AI for insights had a hard time making use of existing models in production.

Natural Language Processing has been one of the most researched fields in deep learning in 2020, mostly due to its rising popularity, future potential, and support for a wide variety of applications. If you have played around with deep learning before, you probably know conventional deep learning frameworks such as Tensorflow, Keras, and Pytorch. Assuming that you know these basic frameworks, this tutorial is dedicated to briefly guide you with other useful NLP libraries that you can learn and use in 2020. Depending on what you want to do, you might be able to take away a few names of the tools that interest you or didn't know exist!

A novel deep-learning algorithm that would perform the path planning based on the row-wise crop environments. I would discuss some of eh important aspects of those research and the application methodologies for the same. I would move in a less-mathematical way and let the readers (at any level of studies) understand vividly the method implied in the same. Since no such dataset was present for the experimentation, the authors build their own [mentioned in [1, 2]]. They devised an algorithm for generating grids of shape H * W, with a random number of rows, denoted by N (the max value for the rows has been considered 50 and the min to be 20). In addition to the generated datasets, they have used some of the images taken from google satellite images, comprising of the row-based crop scenarios. Those images have been manually processed for the extraction of grids and waypoints.

The recent explosion in applications of machine learning to satellite imagery often rely on visible images and therefore suffer from a lack of data during the night. The gap can be filled by employing available infra-red observations to generate visible images. This work presents how deep learning can be applied successfully to create those images by using U-Net based architectures. The proposed methods show promising results, achieving a structural similarity index (SSIM) up to 86% on an independent test set and providing visually convincing output images, generated from infra-red observations.

You've probably heard us say this countless times: GPT-3, the gargantuan AI that spews uncannily human-like language, is a marvel. You can tell with a simple trick: Ask it the color of sheep, and it will suggest "black" as often as "white"--reflecting the phrase "black sheep" in our vernacular. That's the problem with language models: because they're only trained on text, they lack common sense. Now researchers from the University of North Carolina, Chapel Hill, have designed a new technique to change that. They call it "vokenization," and it gives language models like GPT-3 the ability to "see."