Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization