Existing imitation learning approaches often require that the complete demonstration data, including sequences of actions and states, are available. In this paper, we consider a more realistic and difficult scenario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are unobserved. We propose a novel tensor-based model to infer the unobserved actions of the expert state sequences. The policy of the agent is then optimized via a hybrid objective combining reinforcement learning and imitation learning. We evaluated our hybrid approach on an illustrative domain and Atari games. The empirical results show that (1) the agents are able to leverage state expert sequences to learn faster than pure reinforcement learning baselines, (2) our tensor-based action inference model is advantageous compared to standard deep neural networks in inferring expert actions, and (3) the hybrid policy optimization objective is robust against noise in expert state sequences.

Deep learning for object classification relies heavily on convolutional models. While effective, CNNs are rarely interpretable after the fact. An attention mechanism can be used to highlight the area of the image that the model focuses on thus offering a narrow view into the mechanism of classification. We expand on this idea by forcing the method to explicitly align images to be classified to reference images representing the classes. The mechanism of alignment is learned and therefore does not require that the reference objects are anything like those being classified. Beyond explanation, our exemplar based cross-alignment method enables classification with only a single example per category (one-shot). Our model cuts the 5-way, 1-shot error rate in Omniglot from 2.1% to 1.4% and in MiniImageNet from 53.5% to 46.5% while simultaneously providing point-wise alignment information providing some understanding on what the network is capturing. This method of alignment also enables the recognition of an unsupported class (open-set) in the one-shot setting while maintaining an F1-score of above 0.5 for Omniglot even with 19 other distracting classes while baselines completely fail to separate the open-set class in the one-shot setting.

Although momentum-based optimization methods have had a remarkable impact on machine learning, their heuristic construction has been an obstacle to a deeper understanding. A promising direction to study these accelerated algorithms has been emerging through connections with continuous dynamical systems. Yet, it is unclear whether the main properties of the underlying dynamical system are preserved by the algorithms from which they are derived. Conformal Hamiltonian systems form a special class of dissipative systems, having a distinct symplectic geometry. In this paper, we show that gradient descent with momentum preserves this symplectic structure, while Nesterov's accelerated gradient method does not. More importantly, we propose a generalization of classical momentum based on the special theory of relativity. The resulting conformal symplectic and relativistic algorithm enjoys better stability since it operates on a different space compared to its classical predecessor. Its benefits are discussed and verified in deep learning experiments.

In this work, we connect two distinct concepts for unsupervised domain adaptation: feature distribution alignment between domains by utilizing the task-specific decision boundary and the Wasserstein metric. Our proposed sliced Wasserstein discrepancy (SWD) is designed to capture the natural notion of dissimilarity between the outputs of task-specific classifiers. It provides a geometrically meaningful guidance to detect target samples that are far from the support of the source and enables efficient distribution alignment in an end-to-end trainable fashion. In the experiments, we validate the effectiveness and genericness of our method on digit and sign recognition, image classification, semantic segmentation, and object detection.

With the recent advances in solving large, zero-sum extensive form games, there is a growing interest in the inverse problem of inferring underlying game parameters given only access to agent actions. Although a recent work provides a powerful differentiable end-to-end learning frameworks which embed a game solver within a deep-learning framework, allowing unknown game parameters to be learned via backpropagation, this framework faces significant limitations when applied to boundedly rational human agents and large scale problems, leading to poor practicality. In this paper, we address these limitations and propose a framework that is applicable for more practical settings. First, seeking to learn the rationality of human agents in complex two-player zero-sum games, we draw upon well-known ideas in decision theory to obtain a concise and interpretable agent behavior model, and derive solvers and gradients for end-to-end learning. Second, to scale up to large, real-world scenarios, we propose an efficient first-order primal-dual method which exploits the structure of extensive-form games, yielding significantly faster computation for both game solving and gradient computation. When tested on randomly generated games, we report speedups of orders of magnitude over previous approaches. We also demonstrate the effectiveness of our model on both real-world one-player settings and synthetic data.

This top-down planning approach decides what a good subgoal is before planning to achieve it." "For complex, high-dimensional Markov Decision Processes (MDPs), it may be necessary to represent the policy with function approximation. A problem is mis- specified whenever, the representation cannot express any policy with acceptable performance.

Recently, a PhD researcher, Daniele Grattarola built a framework known as Spektral for mapping relational representation learning which is built in Python and is based on the Keras API. Spektral contains a comprehensive set of tools to build graph neural networks as well as implement some of the popular layers for graph Deep Learning. Spektral is built with semi-supervised deep learning methods for graph data, Graph Neural Network (GNN). According to this paper, the idea of GNN is simple: to encode structural information of the graph, each node v_i can be represented by a low-dimensional state vector s_i, 1 i N. The categorisation of deep learning methods on the graph is shown below The core GNN modules of Spektral are mainly based on Keras and it works with all the different backends offered by Keras in order to promptly start experimenting with the Relational Representation Learning (RRL) which consists of developing models that are able to deal with graphs natively without having to deal with the distracting low-level details. The accessory modules of Spectral are built in Numpy/Scipy and for which everything should work at the speed of almost-C-like and without compatibility issues.

Artificial Intelligence is at peak buzzword: it elicits either the euphoria of a technological paradise with anthropomorphic robots to tidy up after us, or fears of hostile machines breaking the human spirit in a world without hope. The Artificial Intelligences of our reality are those of Machine Learning and Deep Learning. Let's make it simple: both are AI – but not the AI of fiction. Instead, these are limited intelligences capable of only the task they are created for: "weak" or "narrow" AI. Machine Learning is essentially applied Statistics, excellently explained in Hastie and Tibshirani's Introduction to Statistical Learning.

On Valentine's Day, a non-profit research company called OpenAI, gifted us a paper with a blog post that rocked my world as an educator. We've trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization -- all without task-specific training. In other words, they had demonstrated how a language processing AI could learn, from millions of webpages, how to undertake written tasks (some of which are of reasonably high quality in terms of sense and coherence) without specifically being trained to do this via a supervised learning process. To understand the implications of the research, it is worth trying to get to grips with what supervised and unsupervised machine learning is (for someone like me this is a steep learning curve!). In supervised learning problems, we start with a data set containing training examples with associated correct labels.

A lot of news made headlines this week at the third annual TensorFlow Dev Summit. New versions of TensorFlow, including TensorFlow 2.0 with tf.keras as a central API and TensorFlow Lite 1.0 for mobile devices, were released, as was a $150 Coral board for edge TPU applications. Speed optimization for AI on mobile devices and a cleanup of TensorFlow's cluttered APIs is more than cosmetic -- these changes will shape how developers and businesses train AI systems. But the news that caught my eye was the release of TensorFlow for federated learning. TensorFlow Federated will provide distributed machine learning for developers to train models across many mobile devices without data ever leaving those devices.