Review for NeurIPS paper: Adversarial Distributional Training for Robust Deep Learning

Additional Feedback: I thought the method was very cool! One thing that I thought the paper was doing (which turned out to be a misunderstanding, I think) is relaxing the l2 adversarial constraint bit. This is more of an intuition (and did not affect my review in any way), but to some extent is seems like if what one cares about is L2-adversarial robustness, then maximizing the inner loss with PGD is in some sense going to be "optimal"/hard-to-beat (some results in the Madry et al paper corroborate this, few-step PGD is pretty good at finding the best maxima we can find in general.) On the other hand, what you have is a weaker adversary (the distributional one entropic regularizer), but it has the advantage of being a potentially structured way of enforcing a better constraint than L2 robustness. Again this isn't part of my review, but it would be cool to see if it is possible to define a new robustness constraint that is explicitly tailored to your learned adversary (e.g.