Energy
Dr Jekyll and Mr Hyde: The Strange Case of Off-Policy Policy Updates Romain Laroche Microsoft Research Montréal, Canada Rémi T achet des Combes Microsoft Research Montréal, Canada
The policy gradient theorem states that the policy should only be updated in states that are visited by the current policy, which leads to insufficient planning in the off-policy states, and thus to convergence to suboptimal policies. We tackle this planning issue by extending the policy gradient theory to policy updates with respect to any state density. Under these generalized policy updates, we show convergence to optimality under a necessary and sufficient condition on the updates' state densities, and thereby solve the aforementioned planning issue. We also prove asymptotic convergence rates that significantly improve those in the policy gradient literature. To implement the principles prescribed by our theory, we propose an agent, Dr Jekyll & Mr Hyde (J&H), with a double personality: Dr Jekyll purely exploits while Mr Hyde purely explores. J&H's independent policies allow to record two separate replay buffers: one on-policy (Dr Jekyll's) and one off-policy (Mr Hyde's), and therefore to update J&H's models with a mixture of on-policy and off-policy updates. More than an algorithm, J&H defines principles for actor-critic algorithms to satisfy the requirements we identify in our analysis. We extensively test on finite MDPs where J&H demonstrates a superior ability to recover from converging to a suboptimal policy without impairing its speed of convergence. We also implement a deep version of the algorithm and test it on a simple problem where it shows promising results.
NSNet: A General Neural Probabilistic Framework for Satisfiability Problems Zhaoyu Li
NSNet can be flexibly configured to solve both SA T and #SA T problems by applying different learning objectives.
Multiwavelet-based Operator Learning for Differential Equations
The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme.
Constrained Sampling with Primal-Dual Langevin Monte Carlo
To do so, we bring classical optimization arguments for saddle-point algorithms to the geometry of Wasserstein space.
CNN Training data Test image Denoised image! CNN Test image Denoised image! CNN Training data Test image Denoised image!
Here we propose "GainTuning", a
LAVIB: A Large-scale Video Interpolation Benchmark
LA VIB comprises a large collection of high-resolution videos sourced from the web through an automated pipeline with minimal requirements for human verification. Metrics are computed for each video's motion magnitudes, luminance conditions, frame
NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning Rong-Jun Qin
Offline reinforcement learning (RL) aims at learning effective policies from historical data without extra environment interactions.
