Gaussian processes (GPs) can provide a principled approach to uncertainty quantification with easy-to-interpret kernel hyperparameters, such as the lengthscale, which controls the correlation distance of function values. However, selecting an appropriate kernel can be challenging. Deep GPs avoid manual kernel engineering by successively parameterizing kernels with GP layers, allowing them to learn low-dimensional embeddings of the inputs that explain the output data. Following the architecture of deep neural networks, the most common deep GPs warp the input space layer-by-layer but lose all the interpretability of shallow GPs. An alternative construction is to successively parameterize the lengthscale of a kernel, improving the interpretability but ultimately giving away the notion of learning lower-dimensional embeddings.

However, this setting might be unrealistic as different classes of actions often co-occur inpractice.

For all the experiments, we generate augmentations in the same way as in MoCo v2 [1] for pretraining. The learning rate is set tolr = 0.1 and is gradually annealed following a cosine decay schedule [3]. For linear classification, all models are trained for 100 epochs with alearning rate oflr = 10.0. For each image, we randomly generate 32 augmented images and feed these images into the pre-trained network toextract features. The feature vectors are`2 normalized before computing similarities and variances.

In Distributional Reinforcement Learning (D-RL) [Bellemare et al., 2023], an agent aims to estimate Sutton and Barto, 2018], where the objective is to predict the expected return only. In Section 3, we answer this methodological question, showing that it is possible to reformulate Policy Evaluation in a distributional setting so that its performance index is explicitly intertwined with the representation of the (state or action) spaces.