When Does Group Invariant Learning Survive Spurious Correlations?