Models Out of Line: A Fourier Lens on Distribution Shift Robustness