The Sampled Policy Gradient (SPG) algorithm is a new offline actor-critic variant that samples in the action space to approximate the policy gradient. It does so by using the critic to evaluate the sampled actions. SPG offers theoretical promise over similar algorithms such as DPG as it searches the action-Q-value space independently of the local gradient, enabling it to avoid local minima. This paper aims to compare SPG to two similar actor-critic algorithms, CACLA and DPG. The comparison is made across two different environments, two different network architectures, as well as training on on-policy transitions in contrast to using an experience buffer. Results seem to show that although SPG does often not perform the worst, it doesn't always match the performance of the best performing algorithm at a particular task. Further experiments are required to get a better estimate of the qualities of SPG.

Figure 1: A driving scene top view representation with superposed forecast probability density functions represented in blue shades in log scale. The forcasting model uses the past trajectories plotted in gray as input. Abstract -- V ehicle trajectory forecasting models use a wide variety of frameworks for interaction and multi-modality. They rely on various representations of the road scene and definitions of maneuvers. In this paper we present a simple model that simultaneously forecasts each vehicle position on a road scene as a sequence of multi-modal probability density functions. This relies solely on vehicle position tracks and does not define maneuvers. We produce an easily extendable model that combines these predictive capabilities while surpassing state-of-the-art results. Its architecture uses multi-head attention to account for complete interactions between all vehicles, and long short-term memory (LSTM) layers for encoding and forecasting. I. INTRODUCTION Automation of driving tasks aims for safety and comfort improvements. For that purpose, most Autonomous Driving (AD) system relies on the anticipation of the traffic scene movements.

What makes a task relatively more or less difficult for a machine compared to a human? Much AI/ML research has focused on expanding the range of tasks that machines can do, with a focus on whether machines can beat humans. Allowing for differences in scale, we can seek interesting (anomalous) pairs of tasks T, T'. We define interesting in this way: The "harder to learn" relation is reversed when comparing human intelligence (HI) to AI. While humans seems to be able to understand problems by formulating rules, ML using neural networks does not rely on constructing rules. We discuss a novel approach where the challenge is to "perform well under rules that have been created by human beings." We suggest that this provides a rigorous and precise pathway for understanding the difference between the two kinds of learning. Specifically, we suggest a large and extensible class of learning tasks, formulated as learning under rules. With these tasks, both the AI and HI will be studied with rigor and precision. The immediate goal is to find interesting groundtruth rule pairs. In the long term, the goal will be to understand, in a generalizable way, what distinguishes interesting pairs from ordinary pairs, and to define saliency behind interesting pairs. This may open new ways of thinking about AI, and provide unexpected insights into human learning.

Understanding latent user needs beneath shopping behaviors is critical to e-commercial applications. Without a proper definition of user needs in e-commerce, most industry solutions are not driven directly by user needs at current stage, which prevents them from further improving user satisfaction. Representing implicit user needs explicitly as nodes like "outdoor barbecue" or "keep warm for kids" in a knowledge graph, provides new imagination for various e- commerce applications. Backed by such an e-commerce knowledge graph, we propose a supervised learning algorithm to conceptualize user needs from their transaction history as "concept" nodes in the graph and infer those concepts for each user through a deep attentive model. Offline experiments demonstrate the effectiveness and stability of our model, and online industry strength tests show substantial advantages of such user needs understanding.

Riemannian Motion Policy Fusion through Learnable Lyapunov Function Reshaping Mustafa Mukadam 1, Ching-An Cheng 1, Dieter Fox 2,3, Byron Boots 2,3, and Nathan Ratliff 3 1 Georgia Institute of Technology, USA 2 University of Washington, USA 3 NVIDIA, USA Abstract: RMPflow is a recently proposed policy-fusion framework based on differential geometry. While RMPflow has demonstrated promising performance, it requires the user to provide sensible subtask policies as Riemannian motion policies (RMPs: a motion policy and an importance matrix function), which can be a difficult design problem in its own right. We propose RMPfusion, a variation of RMPflow, to address this issue. RMPfusion supplements RMPflow with weight functions that can hierarchically reshape the Lyapunov functions of the subtask RMPs according to the current configuration of the robot and environment. This extra flexibility can remedy imperfect subtask RMPs provided by the user, improving the combined policy's performance. These weight functions can be learned by back-propagation. Moreover, we prove that, under mild restrictions on the weight functions, RMPfusion always yields a globally Lyapunov-stable motion policy. This implies that we can treat RMPfusion as a structured policy class in policy optimization that is guaranteed to generate stable policies, even during the immature phase of learning. We demonstrate these properties of RMPfusion in imitation learning experiments both in simulation and on a real-world robot. Keywords: Reactive motion generation, Structured end-to-end learning 1 Introduction Motion planning and control are core techniques to robotics [1, 2, 3]. Ideally a good algorithm must be both computationally efficient and capable of navigating a robot safely and stably across a wide range of environments. Several systems were recently proposed to address this challenge [4, 5, 6] through closely integrating planning and control techniques. In particular, RMPflow [6] is designed to combine reactive policies [7, 8, 9, 10, 11] and planning [12]. Based on differential geometry, RMPflow offers a unified treatment of the nonlinear geometries arising from a robot's internal kinematics and task spaces (e.g.

Online advertising in E-commerce platforms provides sellers an opportunity to achieve potential audiences with different target goals. Ad serving systems (like display and search advertising systems) that assign ads to pages should satisfy objectives such as plenty of audience for branding advertisers, clicks or conversions for performance-based advertisers, at the same time try to maximize overall revenue of the platform. In this paper, we propose an approach based on linear programming subjects to constraints in order to optimize the revenue and improve different performance goals simultaneously. We have validated our algorithm by implementing an offline simulation system in Alibaba E-commerce platform and running the auctions from online requests which takes system performance, ranking and pricing schemas into account. We have also compared our algorithm with related work, and the results show that our algorithm can effectively improve campaign performance and revenue of the platform.

Over the weekend, bizarre footage of a facial projection technology circulated on social media in connection with the ongoing protests in Hong Kong. The mysterious device shows a headband with a large digital projector, which projects a digital image of another person's face onto whoever is wearing the device. The device was assumed to be a countermeasure to the recent ban on face coverings in Hong Kong. Initially images from the 2017 art project were thought to come from the Hong Kong protests. In fact, the videos came from a 2017 art project called'Anonymous,' created by students at Utrecht School of the Arts in the Netherlands.

Researchers at the University of Washington have made an important breakthrough in robotic suction, a simple concept that's been difficult to master. The scientists took inspiration from the Northern clingfish, a species common in the Pacific Northwest known for clinging to the underside of slipper oceanic rocks. The fish has a suction cup on its belly, which uses small hairlike structures to create a powerful connection to the slipperiest surfaces. The Nothern clingfish (pictured above) is famous for being able to attach to the slipperiest and most uneven surfaces, something which made it a main focus of researchers. A half-pound fish can create suction strong enough to lift a rock twelve times its bodyweight, and the connection is so powerful it remains in tact even after the fish has died.

Fake news is a well-documented problem. It's obvious that an AI-based technique is needed to counter that threat. AI-generated videos (also known as deepfakes) and images are easier to come up with than ever before. They can range from funny and quirky to much more sinister and dangerous, such as those of political statements that were never given, or shots of events that never took place. It's easy to see how such content published on social networks or by reputable media outlets could be used to manipulate public opinion.

Illustration by Belkin et al. (2018) of the effect of increased model complexity on generalization: traditional belief (a) vs actual practice (b). Traditional wisdom in machine learning holds that there is a careful trade-off between training error and generalization gap. There is a "sweet spot" for the model complexity such that the model (i) is big enough to achieve reasonably good training error, and (ii) is small enough so that the generalization gap – the difference between test error and training error – can be controlled. A smaller model would give a larger training error while making the model bigger would result in a larger generalization gap, both leading to larger test errors. This is described by the classical U-shaped curve for the test error when the model complexity varies (see Figure 1(a)).