Residual Alignment: Uncovering the Mechanisms of Residual Networks

Neural Information Processing Systems 

C is the number of classes; and (RA4) top singular values of Residual Jacobians scale inversely with depth.