Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

We would like to point out that K-means is used only once to17 initialize the representativesets and isnot anintrinsic component ofthe online algorithm. What is important to observe21 is that N is kept constant throughout in order to reduce the storage footprint and to ensure low-complexity online22 processing. Tomaintain the list constant, for every added point another point is removed. Also, the reviewer is correct24 in observing thatN does not feature in the convergence results, which are asymptotic and do not imply anything25 about the convergence rate. Clearly, if the point dimensionm is large, it is beneficial to increaseN.

Many of these algorithms have been successfully used with specific loss functions such as the Hamming loss. Their use has been also extended to multivariate performance measures such as Precision/Recall orF1-score (Joachims,2005),which depend onpredictions onalltraining points.

However, state-of-the-art (SOT A) RL techniques with Temporal-Difference (TD) learning frameworks often suffer from learning inefficiency, due to the significant expansion of the augmented state space with the delay. To improve learning efficiency without sacrificing performance, this work introduces a novel framework called V ariational Delayed Policy Optimization (VDPO), which reformulates delayed RL as a variational inference problem. This problem is further modelled as a two-step iterative optimization problem, where the first step is TD learning in the delay-free environment with a small state space, and the second step is behaviour cloning which can be addressed much more efficiently than TD learning. We not only provide a theoretical analysis of VDPO in terms of sample complexity and performance, but also empirically demonstrate that VDPO can achieve consistent performance with SOT A methods, with a significant enhancement of sample efficiency (approximately 50% less amount of samples) in the MuJoCo benchmark.

Neural Information Processing Systems http://nips.cc/

Related mixed matrix-tensor models have also been studied in the context of text-analysis applications in [20, 19].