In Distributional Reinforcement Learning (D-RL) [Bellemare et al., 2023], an agent aims to estimate Sutton and Barto, 2018], where the objective is to predict the expected return only. In Section 3, we answer this methodological question, showing that it is possible to reformulate Policy Evaluation in a distributional setting so that its performance index is explicitly intertwined with the representation of the (state or action) spaces.

DIP finds a solution without needing any data other than the limited/corrupted task-specific measurements and the forward operator.

Wedesigned to: 1) answer anchors 5.2); 2) executed variables graphs ofmillions 5.1 Setup Dataset.

It allows for both latent confounders and selection variables, although inthis paper we assume the absence of selection variables.

Inthemuch more complex football game, URI's policy beat the built-in AIs with a 37% winning rate while GPT-based agents can only achieve a 6% winning rate.

Inthissetting,all rewards drawn from an arm are independent and identically distributed.