Our experiments with extensive data indicate that the model trained on generated data of the advanced model can approximate its generation capability. However, it requires large-scale samples of 10 million or more.

Reward specification is one of the most tricky problems in Reinforcement Learning, which usually requires tedious hand engineering in practice.

We develop an adaptive strategy to design indirect experiments that optimally inform a targeted query about the ground truth mechanism in terms of sequentially narrowing the gap between an upper and lower bound on the query.

Additionally, a hash-based representation is proposed for memory-efficient spherical frustum storage.