Smoothly Giving up: Robustness for Simple Models