Sparse maximal update parameterization: A holistic approach to sparse training dynamics

May-29-2025, 05:01:19 GMT–Neural Information Processing Systems 

Several challenges make it difficult for sparse neural networks to compete with dense models.