The slate recommendation problem aims to find the "optimal" ordering of a subset of documents to be presented on a surface that we call "slate". The definition of "optimal" changes depending on the underlying applications but a typical goal is to maximize user engagement with the slate. Solving this problem at scale is hard due to the combinatorial explosion of documents to show and their display positions on the slate. In this paper, we introduce Slate Conditional Variational Auto-Encoders (Slate-CVAE) to generate optimal slates. To the best of our knowledge, this is the first conditional generative model that provides a unified framework for slate recommendation by direct generation. Slate-CVAE automatically takes into account the format of the slate and any biases that the representation causes, thus truly proposing the optimal slate. Additionally, to deal with large corpora of documents, we present a novel approach that uses pretrained document embeddings combined with a soft-nearest-neighbors layer within our CVAE model. Experiments show that on the simulated and real-world datasets, Slate-CVAE outperforms recommender systems that consists of greedily ranking documents by a significant margin while remaining scalable.

Exploring why stochastic gradient descent (SGD) based optimization methods train deep neural networks (DNNs) that generalize well has become an active area of research. Towards this end, we empirically study the dynamics of SGD when training over-parametrized DNNs. Specifically we study the DNN loss surface along the trajectory of SGD by interpolating the loss surface between parameters from consecutive \textit{iterations} and tracking various metrics during training. We find that the loss interpolation between parameters before and after a training update is roughly convex with a minimum (\textit{valley floor}) in between for most of the training. Based on this and other metrics, we deduce that during most of the training, SGD explores regions in a valley by bouncing off valley walls at a height above the valley floor. This 'bouncing off walls at a height' mechanism helps SGD traverse larger distance for small batch sizes and large learning rates which we find play qualitatively different roles in the dynamics. While a large learning rate maintains a large height from the valley floor, a small batch size injects noise facilitating exploration. We find this mechanism is crucial for generalization because the valley floor has barriers and this exploration above the valley floor allows SGD to quickly travel far away from the initialization point (without being affected by barriers) and find flatter regions, corresponding to better generalization.

We describe a mechanism by which artificial neural networks can learn rapid adaptation - the ability to adapt on the fly, with little data, to new tasks - that we call conditionally shifted neurons. We apply this mechanism in the framework of metalearning, where the aim is to replicate some of the flexibility of human learning in machines. Conditionally shifted neurons modify their activation values with task-specific shifts retrieved from a memory module, which is populated rapidly based on limited task experience. On metalearning benchmarks from the vision and language domains, models augmented with conditionally shifted neurons achieve state-of-the-art results.

The autoencoder is an artificial neural network that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar to the eigenvalues in PCA that are paired with eigenvectors. We propose a novel supervised node saliency (SNS) method that ranks the hidden nodes, which contain weight vectors for transformations. SNS is able to indicate the nodes specialized in a learning task. The latent representations of a hidden node can be described using a one-dimensional histogram. We apply normalized entropy difference (NED) to measure the "interestingness" of the histograms, and conclude a property for NED values to identify a good classifying node. Keywords: networks, node selection Autoencoder, latent representations, unsupervised learning, neural By 1. Background and Motivation The autoencoder is an artificial neural network model that aims to find an encoding for a dataset in a reduced dimension [1]. The model is unsupervised because class labels (i.e. The encoding of autoencoders constructs a powerful representation and often learns useful properties of the data [2, 3]. The unsupervised feature extraction provided by the encoding of autoencoders is a key factor in the success of pattern recognition [2, 4, 5, 6, 7, 8, 9]. For example, theoretical studies [10] suggest that we may need deep architectures to efficiently model complex distributions and obtain better performance on challenging pattern recognition tasks. Training the autoencoders becomes a successful approach to solving the difficult optimization problem, which arises from building a multi-layer neural network [11, 12, 13, 10].

Understanding the flow of information in Deep Neural Networks (DNNs) is a challenging problem that has gain increasing attention over the last few years. While several methods have been proposed to explain network predictions, there have been only a few attempts to compare them from a theoretical perspective. What is more, no exhaustive empirical comparison has been performed in the past. In this work, we analyze four gradient-based attribution methods and formally prove conditions of equivalence and approximation between them. By reformulating two of these methods, we construct a unified framework which enables a direct comparison, as well as an easier implementation. Finally, we propose a novel evaluation metric, called Sensitivity-n and test the gradient-based attribution methods alongside with a simple perturbation-based attribution method on several datasets in the domains of image and text classification, using various network architectures.

The Shadow Robot Company, that manufactures robotic hands for grasping and manipulation for real world challenges from fruit picking to bomb disposal, is supplying its Shadow Dexterous Hands to OpenAI, a non-profit company focusing on the path to safe artificial intelligence. The research is claimed to have created eight newly released environments, four of which using the Shadow Hand robot to solve realistic manipulation tasks. The Shadow Hand is tactile enough to rotate a block and a solid egg and flexible enough to move a pen between its fingers. Each task has a'goal', such as achieving the desired orientation of a block in the Shadow hand block manipulation task. Along with releasing these new robotics environments, OpenAI is releasing code for Hindsight Experience Replay, a reinforcement learning algorithm that can teach and improve robotic technology to learn from failure.

GOOGLE HAS announced that it is making its artificial intelligence (AI) and machine learning (ML) courses available to everyone, beyond Mountain View. The new Learn with Google AI portal will let anyone with an interest, at almost any level, learn how to make the most of the glorious new horizon of AI and neural networking. Previously, the courses were designed for internal use to train Googlers, as Google continues its moves to be an "AI First" company. Zuri Kemp from Google AI explains: "From deep learning experts looking for advanced tutorials and materials on TensorFlow, to "curious cats" who want to take their first steps with AI, anyone looking for educational content from ML experts at Google can find it here." There's also a Machine Learning Crash Course (MLCC) which "provides exercises, interactive visualizations, and instructional videos that anyone can use to learn and practice ML concepts."

The NVIDIA Deep Learning Institute (DLI) will show you the latest techniques in designing, training, and deploying neural network-powered machine learning in your applications. Perks like credits and exclusive courses are also available to Inception members.

Every decade seems to have its technological buzzwords: we had personal computers in 1980s; Internet and worldwide web in 1990s; smart phones and social media in 2000s; and Artificial Intelligence (AI) and Machine Learning in this decade. As mentioned in a previous article [56], the 1950-82 era saw a new field of Artificial Intelligence (AI) being born, lot of pioneering research being done, massive hype being created but eventually fizzling out. The 1983-2004 era saw research and development in AI gradually picking up and leading to a few key accomplishments (e.g., Deep Blue beating Kasparov in Chess) and commercialized solutions (e.g., Cyberknife), but its pace really picked up during 2005 and 2010 [57]. Since 2011, AI research and development has been witnessing hypergrowth, and researchers have created several AI solutions that are almost as good as – or better than – humans in several domains; these include playing games, healthcare, computer vision and object recognition, speech to text conversion, speaker recognition, and improved robots and chat-bots for solving specific problems. The table in the Appendix lists key AI solutions that are rivaling humans in various domains and six of these solutions are described below.

The course is a new one by Andrew Ng, Co-founder, Coursera; Adjunct Professor, Stanford University; formerly head of Baidu AI Group/Google Brain. It will start Aug 15. About this course: If you want to break into cutting-edge AI, this course will help you do so. Deep learning engineers are highly sought after, and mastering deep learning will give you numerous new career opportunities. Deep learning is also a new "superpower" that will let you build AI systems that just weren't possible a few years ago.