Second, weintroduceColosseum,apioneering package thatenables empirical hardness analysis and implements a principled benchmark composed of environments that are diverse with respect to different measures of hardness.

Figure 5: State Supportfor RPBBAC (left) and SAC (right) in MountainCar -Continuous-v0.

Real-world reinforcement learning (RL) applications often come with possibly infinite state and action space, and in such a situation classical RL algorithms developed in the tabular setting are not applicable anymore. A popular approach to overcoming this issue is by applying function approximation techniques to the underlying structures of the Markovdecision processes (MDPs).