Review for NeurIPS paper: Consequences of Misaligned AI

Weaknesses: The theoretical setting makes quite strong assumptions, and doesn't really discuss the intuition behind them, so it would be easy for a cursory reader to infer that more is happening than really is. In particular, the various component-wise strict increase assumptions are doing a lot of work. Here are the various results translated into prose: Theorem 1: If moving in a particular direction D strictly increases the utility available from moving in other directions, an optimal agent will move as far as possible along D. Theorem 2: The only way moving arbitrarily far can't arbitrarily decrease utility is if one can move arbitrarily far without arbitrarily decreasing utility. Proposition 1: We can decrease utility by moving arbitrarily far if the boundary shape vs. utility slope has a certain shape. Proposition 2: If we fix some dimensions, we can compute utility ignoring the fixed dimensions. Proposition 3: An agent that is allowed to move arbitrarily far in one step is basically the same as a non-interactive agent.