Artificial intelligence (AI) systems possess significant potential to drive societal progress. However, their deployment often faces obstacles due to substantial safety concerns.

Conditioning on the current game context, the generation model predicts the distribution of the currently acting player.

In a single-agent setting, IL was shown to be done efficiently via an inverse soft-Q learning process.

Reward functions are difficult to design and often hard to align with human intent.

Here we present the proofs for the arguments behind each of the approximations described in Section4.