Reviews: Controlling Neural Level Sets

This paper addresses the important task of controlling the level sets that comprise the decision boundaries of a neural network. I think the proposed method is quite reasonable, well-described, and convincingly demonstrated to be quite useful across a number of tasks. Re level set sampling: - Doesn't the ReLU activation imply that D_x F(p; theta) is often 0 at many points p? How do you get around this issue when attempting to optimize p toward S(theta)? It seems this optimization might often get stuck in regions where D_x F(p;theta) 0 yet p lies far away from S(theta). Furthermore, the particular choice used by the authors should be a bit better motivated in the text as it's not clear to me. - It seems one can sample from level 0 set by instead just optimizing: min_x F(x;theta) via gradient descent in x.