Visual Reinforcement Learning (RL) agents trained on limited views face significant challenges in generalizing their learned abilities to unseen views.

We introduce Policy Optimization with Multiple Optima (POMO), anend-to-end approach forbuildingsuchaheuristic solver.POMO isapplicable to a wide range of CO problems.

Specifically, instead of directly measuring the divergence with paired images, we train a reward model with the dataset we construct, consisting of nearly 51,000 images annotated with human preferences.

Several works haveshown this vulnerability via adversarial attacks, butexisting approaches onimproving therobustness ofDRL under this setting have limited success and lack for theoretical principles. We show that naively applying existing techniques on improving robustness for classification tasks,likeadversarialtraining,areineffectiveformanyRLtasks.

Given the impressive results of deepreinforcement learning, weargueforaneedtomoreclearly understand the challenges inthis setting.

The general concept is to represent the training loss as a function of both the agent parameters and the hyperparameters. The agent optimizes the parameters to minimize this loss function, w.r.t the current hyperparameters.

Policy Space Response Oracles (PSRO) is a reinforcement learning (RL) algorithm for two-player zero-sum games that has been empirically shown to find approximate Nash equilibria in large games.