The Risks of Invariant Risk Minimization