The Graph Convolutional Networks (GCN) has demonstrated superior performance in representing graph data, especially homogeneous graphs. However, the real-world graph data is usually heterogeneous and evolves with time, e.g., Facebook and DBLP, which has seldom been studied. To cope with this issue, we propose a novel approach named temporal heterogeneous graph convolutional network (THGCN). THGCN first embeds both spatial information and node attribute information together. Then, it captures short-term evolutionary patterns from the aggregations of embedded graph signals through compression network. Meanwhile, the long-term evolutionary patterns of heterogeneous graph data are also modeled via a TCN temporal convolutional network. To the best of our knowledge, this is the first attempt to model temporal heterogeneous graph data with a focus on community discovery task.

For dynamical systems that can be modelled as asymptotically stable linear systems forced by Gaussian noise, this paper develops methods to infer their modes from observations in real time. The modes can be real or complex. For a real mode, we infer its damping rate, mode shape and amplitude. For a complex mode, we infer its frequency, damping rate, (complex) mode shape and (complex) amplitude. The work is motivated and illustrated by the problem of detection of oscillations in power flow in AC electrical networks. Suggestions of other applications are given.

Decentralized stochastic gradient method emerges as a promising solution for solving large-scale machine learning problems. This paper studies the decentralized Markov chain gradient descent (DMGD) algorithm - a variant of the decentralized stochastic gradient methods where the random samples are taken along the trajectory of a Markov chain. This setting is well-motivated when obtaining independent samples is costly or impossible, which excludes the use of the traditional stochastic gradient algorithms. Specifically, we consider the first- and zeroth-order versions of decentralized Markov chain gradient descent over a connected network, where each node only communicates with its neighbors about intermediate results. The nonergodic convergence and the ergodic convergence rate of the proposed algorithms have been rigorously established, and their critical dependences on the network topology and the mixing time of Markov chain have been highlighted. The numerical tests further validate the sample efficiency of our algorithm.

There has been an emerging trend in non-Euclidean dimension reduction of aiming to recover a low dimensional structure, namely a manifold, underlying the high dimensional data. Recovering the manifold requires the noise to be of certain concentration. Existing methods address this problem by constructing an output manifold based on the tangent space estimation at each sample point. Although theoretical convergence for these methods is guaranteed, either the samples are noiseless or the noise is bounded. However, if the noise is unbounded, which is a common scenario, the tangent space estimation of the noisy samples will be blurred, thereby breaking the manifold fitting. In this paper, we introduce a new manifold-fitting method, by which the output manifold is constructed by directly estimating the tangent spaces at the projected points on the underlying manifold, rather than at the sample points, to decrease the error caused by the noise. Our new method provides theoretical convergence, in terms of the upper bound on the Hausdorff distance between the output and underlying manifold and the lower bound on the reach of the output manifold, when the noise is unbounded. Numerical simulations are provided to validate our theoretical findings and demonstrate the advantages of our method over other relevant methods. Finally, our method is applied to real data examples.

Personal use of this material is permitted. Abstract -- One typical assumption in inverse reinforcement learning (IRL) is that human experts act to optimize the expected utility of a stochastic cost with a fixed distribution. Risk-sensitive inverse reinforcement learning (RS-IRL) bridges such gap by assuming that humans act according to a random cost with respect to a set of subjectively distorted distributions instead of a fixed one. Such assumption provides the additional flexibility to model human's risk preferences, represented by a risk envelope, in safe-critical tasks. However, like other learning from demonstration techniques, RS-IRL could also suffer inefficient learning due to redundant demonstrations. Inspired by the concept of active learning, this research derives a probabilistic disturbance sampling scheme to enable an RS-IRL agent to query expert support that is likely to expose unrevealed boundaries of the expert's risk envelope. Experimental results confirm that our approach accelerates the convergence of RS-IRL algorithms with lower variance while still guaranteeing unbiased convergence. Inverse reinforcement learning (IRL) provides a novel framework for recovering cost functions utilized in human decision making [1]-[6]. The original IRL algorithms [1], [2] are formed as linear programming constrained by op-timality conditions [7]. More recent advancements in IRL include the guided cost learning algorithm [10] which combines MaxEnt IRL and deep learning techniques. The flexibility of IRL framework has prompted its application to a variety of tasks such as autonomous helicopter aerobatics [11] and robot locomotion [12].

In the Bayesian analysis of signals immersed in noise [1], we seek a representation for the posterior probability of one or more parameters that govern the shape of the signals. Unless the parameter-to-signal map (the forward model) is very simple, the analysis (or inverse solution) comes at significant computational cost, as it requires the stochastic exploration of the likelihood surface at a large number of locations in parameter space. Such is the case, for instance, of parameter estimation for gravitational-wave sources such as the compact binaries detected by LIGO-Virgo [2, 3]; here each likelihood evaluation requires that we generate the gravitational waveform corresponding to a set of source parameters, and compute its noise-weighted correlation with detector data [4]. Waveform generation is usually the costlier operation, so gravitational-wave analysts often utilize faster, less accurate waveform models [5, 6], or accelerated surrogates of slower, more accurate models [7]. Extending the analysis from the data we have to the data we might measure (i.e., characterizing the parameter-estimation prospects of future experiments) compounds the expense, since we need to explore posteriors for many noise realizations, and across the domain of possible source parameters. For concreteness, we price the evaluation of a single Bayesian posterior at null 10 6 times the cost of generating a waveform, and the characterization of parameter-estimation prospects at null 10 6 times the cost of a posterior. With current computational resources, this means that (for instance) accurate component-mass estimates only become available hours or days after the detection of a binary black-hole coalescence [8, 9], while any extensive study of parameter-estimation prospects must rely on less reliable techniques such as the Fisher-matrix approximation [10]. In this Letter, we show how one-or two-dimensional marginalized Bayesian posteriors may be produced using deep neural networks [11] trained on large ensembles of signal noise data streams.

Recurrent Neural Networks (RNN) have become competitive forecasting methods, as most notably shown in the winning method of the recent M4 competition. However, established statistical models such as ETS and ARIMA gain their popularity not only from their high accuracy, but they are also suitable for non-expert users as they are robust, efficient, and automatic. In these areas, RNNs have still a long way to go. We present an extensive empirical study and an open-source software framework of existing RNN architectures for forecasting, that allow us to develop guidelines and best practices for their use. For example, we conclude that RNNs are capable of modelling seasonality directly if the series in the dataset possess homogeneous seasonal patterns, otherwise we recommend a deseasonalization step. Comparisons against ETS and ARIMA demonstrate that the implemented (semi-)automatic RNN models are no silver bullets, but they are competitive alternatives in many situations.

A framework to boost efficiency of Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation, which we call the refined variational approximation. Its strength lies both in ease of implementation and in automatically tuning the sampler parameters to speed up mixing time. Several strategies to approximate the \emph{evidence lower bound} (ELBO) computation are introduced, including a rewriting of the ELBO objective. A specialization towards state-space models is proposed. Experimental evidence of its efficient performance is shown by solving an influence diagram in a high-dimensional space using a conditional variational autoencoder (cVAE) as a deep Bayes classifier; an unconditional VAE on density estimation tasks; and state-space models for time-series data.

Sizhong Lan 1 1 China Mobile Research Institute, Beijing 100053, China a) (Dated: September 24, 2019) It had been proved that every non-cooperative game had a Nash equilibrium point. Although many existing algorithms are capable of finding equilibrium points, it is still unclear what force is driving the players to them in the real world. We show that, the players' immediately and constantly pursuing profitable strategies is sufficient for the game to evolve towards equilibrium point, and meanwhile the game needs minimum information exchange among players and no mediation from beyond players. Accordingly, we suggest that in reality the tendency towards Nash equilibrium could be more pervasive and irresistible than expected. Technically, the players' pursuit of profitable strategies gives rise to a sequence of adjusted strategies for our study its approximation to the true equilibrium point. And the sequence can be nicely visualized as a clear path towards an equilibrium point. Our theory has limitations in optimizing the accuracy of equilibrium point approximation. I. INTRODUCTION In 1951 John Nash proved that every non-cooperative game has an equilibrium point 1 by using Brouwer's fixed point theorem 2 . Nash's proof is existential for equilibrium point and yet non-constructive for finding one.

We propose an actor-critic, model-free, and online Reinforcement Learning (RL) framework for continuous-state continuous-action Markov Decision Processes (MDPs) when the reward is highly sparse but encompasses a high-level temporal structure. We represent this temporal structure by a finite-state machine and construct an on-the-fly synchronised product with the MDP and the finite machine. The temporal structure acts as a guide for the RL agent within the product, where a modular Deep Deterministic Policy Gradient (DDPG) architecture is proposed to generate a low-level control policy. We evaluate our framework in a Mars rover experiment and we present the success rate of the synthesised policy.