Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests