In many sequential decision making tasks, it is challenging to design reward functions that help an RL agent efficiently learn behavior that is considered good by the agent designer. A number of different formulations of the reward-design problem, or close variants thereof, have been proposed in the literature. In this paper we build on the Optimal Rewards Framework of Singh et.al. that defines the optimal intrinsic reward function as one that when used by an RL agent achieves behavior that optimizes the task-specifying or extrinsic reward function. Previous work in this framework has shown how good intrinsic reward functions can be learned for lookahead search based planning agents. Whether it is possible to learn intrinsic reward functions for learning agents remains an open problem. In this paper we derive a novel algorithm for learning intrinsic rewards for policy-gradient based learning agents. We compare the performance of an augmented agent that uses our algorithm to provide additive intrinsic rewards to an A2C-based policy learner (for Atari games) and a PPO-based policy learner (for Mujoco domains) with a baseline agent that uses the same policy learners but with only extrinsic rewards. Our results show improved performance on most but not all of the domains.

Deep probabilistic programming languages try to combine the advantages of deep learning with those of probabilistic programming languages. If successful, this would be a big step forward in machine learning and programming languages. Unfortunately, as of now, this new crop of languages is hard to use and understand.

In one of the most common settings in statistical relational learning (SRL), we are given a single relational structure from which we need to learn a model, and this model is then used to make predictions on previously unseen structures. For example, the global relational structure could correspond to a large social network, with the training data specifying the relationships that hold among a small subset of the users, along with their attributes. Clearly, in order to provide any guarantees on the accuracy of these predictions, we need to make (simplifying) assumptions about how the training and test structures are related. In this paper, we follow the setting from [10, 9], where it is assumed that these structures, which we will call relational examples, are all obtained by sampling domain elements from a larger global structure (uniformly and without replacement).

We study feature propagation on graph, an inference process involved in graph representation learning tasks. It's to spread the features over the whole graph to the $t$-th orders, thus to expand the end's features. The process has been successfully adopted in graph embedding or graph neural networks, however few works studied the convergence of feature propagation. Without convergence guarantees, it may lead to unexpected numerical overflows and task failures. In this paper, we first define the concept of feature propagation on graph formally, and then study its convergence conditions to equilibrium states. We further link feature propagation to several established approaches such as node2vec and structure2vec. In the end of this paper, we extend existing approaches from represent nodes to edges (edge2vec) and demonstrate its applications on fraud transaction detection in real world scenario. Experiments show that it is quite competitive.

We propose a simple, robust, and scalable reverse image search engine that leverages convolutional features from Keras' pre-trained neural networks and the distance metric from Scikit-Learn's K-Nearest Neighbors. We show example queries using data scraped from Google images, and dive deeper in how we use the search engine to track the proliferation of memes from the dark web.

Thermal cameras like FLIR, or Forward Looking Infrared, sensors are actively deployed on aerial and ground vehicles, in watch towers and at check points for surveillance purposes. More recently, thermal cameras are becoming available for use as body-worn cameras. The ability to perform automatic face recognition at nighttime using such thermal cameras is beneficial for informing a Soldier that an individual is someone of interest, like someone who may be on a watch list. The motivations for this technology -- developed by Drs. Benjamin S. Riggan, Nathaniel J. Short and Shuowen "Sean" Hu, from the U.S. Army Research Laboratory -- are to enhance both automatic and human-matching capabilities.

Are you interested to explore advanced algorithm concepts such as random forest vector machine, K- nearest, and more through real-world examples? Packt's Video Learning Paths are a series of individual video products put together in a logical and stepwise manner such that each video builds on the skills learned in the video before it. Machine learning and data science are some of the top buzzwords in the technical world today. Machine learning - the application and science of algorithms that makes sense of data, is the most exciting field of all the computer sciences! It explores the study and construction of algorithms that can learn from and make predictions on data.

This is tough for five-year-olds, but I'll give it a shot for ten-year-olds. Like a lot of other machine learning algorithms, SVMs take some data to start with that's already classified (the training set), and tries to predict a set of unclassified data (the testing set). The data that we have often has a lot of different features, and so we can end up plotting each data item as a point in space, with the value of each feature being the value at a particular coordinate. Now (for two data features) what we want to do is find some line that splits the data between the two differently classified groups of data as well as we can. This will be the line such that the distances from the closest point in each of the two groups will be farthest away.

This is the first article of a five-part series about machine learning. Machine learning is a very hot topic for many key reasons, and because it provides the ability to automatically obtain deep insights, recognize unknown patterns, and create high performing predictive models from data, all without requiring explicit programming instructions. Despite the popularity of the subject, machine learning's true purpose and details are not well understood, except by very technical folks and/or data scientists. This series is intended to be a comprehensive, in-depth guide to machine learning, and should be useful to everyone from business executives to machine learning practitioners. It covers virtually all aspects of machine learning (and many related fields) at a high level, and should serve as a sufficient introduction or reference to the terminology, concepts, tools, considerations, and techniques of the field.

In a dynamic network, the neighborhood of the vertices evolve across different temporal snapshots of the network. Accurate modeling of this temporal evolution can help solve complex tasks involving real-life social and interaction networks. However, existing models for learning latent representation are inadequate for obtaining the representation vectors of the vertices for different time-stamps of a dynamic network in a meaningful way. In this paper, we propose latent representation learning models for dynamic networks which overcome the above limitation by considering two different kinds of temporal smoothness: (i) retrofitted, and (ii) linear transformation. The retrofitted model tracks the representation vector of a vertex over time, facilitating vertex-based temporal analysis of a network. On the other hand, linear transformation based model provides a smooth transition operator which maps the representation vectors of all vertices from one temporal snapshot to the next (unobserved) snapshot-this facilitates prediction of the state of a network in a future time-stamp. We validate the performance of our proposed models by employing them for solving the temporal link prediction task. Experiments on 9 real-life networks from various domains validate that the proposed models are significantly better than the existing models for predicting the dynamics of an evolving network.