Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations