Given such a setup, a natural question to ask is how does one measure the performance of the learner? Classical online learning studies one such notion of performance known as regret.

The experiments to produce the figures of the paper were performed on a Linux-based (CentOS) desktop with Intel Xeon CPU E5-2643 v4 @ 3.40GHz (6 cores) and 128 GB of RAM. To produce Figs. 4 and 5 (also supporting Figs. Requirements: python, matplotlib, numpy, scipy The hyperparameters obtained with optuna [2] are provided in the subdirectory results. To recreate a figure run the corresponding script. Bio-OFC is an instance of such a controller.

Policy Learning (ConPoLe) that explicitly optimizes the InfoNCE loss, which lower bounds the mutual information between the current state and next states that continue on a path to the solution.

We perform an extensive evaluation on a broad class of problems: Gaussian modelling, K-nearest neighbors search, geometric deep learning, non-Euclidean embeddings and optimal transport theory.

This work was conducted during Chaobing Song's visit to Professor Yi Ma's group at UC Berkeley.