A Law of Robustness beyond Isoperimetry